MMAU: Multi-Modal Audio Evaluation Benchmark with 12 Task Types

MMAU provides between 1,000 and 10,000 test records for evaluating audio large language models, released by TwinkStart in early 2026. It is integrated into the UltraEval-Audio framework to benchmark performance across 12 task types and 10 languages. The data spans four specialized domains: speech, general sound, medical audio, and music.

Use Cases

• Benchmarking speech understanding using the 'mmau-test-mini' configuration
• Evaluating zero-shot performance on medical audio tasks
• Testing multilingual audio-to-text capabilities across 10 languages

Strengths

• Covers 10 different languages
• Includes 12 distinct task types
• Spans 4 specialized domains including medical and music

Limitations

• Small sample size restricted to the 1K-10K record range
• Test-only data lacks training pairs for model optimization
• Specific column schema is not documented in the metadata

Provenance

Source

TwinkStart / UltraEval-Audio framework

Freshness

Last updated January 2026.

Geography

Global