AISHELL8-RealScene: Mandarin Conversational Speech with Multi-View Video

AISHELL8-RealScene is a multimodal dataset of conversational Mandarin speech recorded in real-world settings. It contains 102.19 hours of audio from 171 foreground speakers across 5 different locations. The dataset was created by SMIIP-lab and includes synchronized near-field and 8-channel far-field audio with multi-view facial video.

Use Cases

• Training far-field automatic speech recognition (ASR) models based on the 8-channel audio recordings.
• Developing audio-visual speech recognition (AVSR) systems based on the synchronized RGB facial video.
• Researching speaker diarization and separation in conversational settings based on the multi-speaker, multi-location recordings.
• Building models for noise-robust speech processing based on real-world acoustic scenes.
• Benchmarking multi-modal fusion techniques for speech using the aligned audio and video streams.

Strengths

• Includes 102.19 hours of conversational speech data.
• Features 171 distinct foreground speakers, providing speaker diversity.
• Captures data from 5 real-world locations, offering environmental variety.
• Provides synchronized multimodal data with 8-channel far-field audio and 256x256 resolution video at 25 fps.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• Data may reflect geographic or acoustic bias inherent to the 5 specific recording locations.

Provenance

Source

SMIIP-lab

Collection Method

Recorded conversational speech in 5 real-world locations with near-field, far-field, and multi-view video capture.

Freshness

Last updated 2026-06-05 04:36:38; freshness should be verified.

Geography

Locations are unspecified but likely within Chinese-speaking regions, given the Mandarin language focus.