MuChoMusic: A Benchmark for Music Understanding in Multimodal Models

MuChoMusic is a benchmark containing 1,187 multiple-choice questions validated by human annotators, based on 644 music tracks from two publicly available datasets. It was created by mulab-mir and last updated on August 5, 2024. The questions cover a wide variety of genres and assess knowledge and reasoning across several musical concepts.

Use Cases

• Benchmarking model performance on music knowledge based on multiple-choice questions.
• Evaluating multimodal reasoning on musical concepts and culture based on audio and text.
• Training models for music understanding tasks based on human-validated question-answer pairs.

Strengths

• Contains 1,187 human-validated multiple-choice questions.
• Based on 644 music tracks from two publicly available datasets.
• Covers a wide variety of music genres and concepts.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.

Provenance

Source

mulab-mir via Hugging Face.

Collection Method

Likely curated from two publicly available music datasets and annotated by human validators.

Freshness

Last updated 2024-08-05 05:54:55.