MIDI Files with Descriptive Text Captions and Musical Features

168,385 MIDI music files are paired with descriptive text captions and a set of extracted musical features. The captions were generated by the amaai-lab using a pipeline that extracts MIR features and employs the Claude 3 LLM for in-context learning captioning. The dataset was last updated in March 2025.

Use Cases

• Train or fine-tune models for generating descriptive text from MIDI files using the extracted musical features as input.
• Develop or evaluate models for retrieving MIDI files based on natural language queries using the text captions.
• Conduct research on the relationship between symbolic musical structures and semantic descriptions via the paired MIDI and caption data.
• Benchmark music information retrieval tasks, such as genre or mood classification, using the provided musical features.

Strengths

• Contains 168,385 MIDI files, providing a large-scale collection for training.
• Includes both descriptive text captions and extracted musical features, offering multimodal data.
• The captioning pipeline and framework are available as open-source code.

Limitations

• The original source and geographic/temporal coverage of the MIDI files are not specified in the provided description.
• Potential for label noise or bias exists as captions are generated by an AI model based on extracted features.
• The dataset size in gigabytes and specific file formats are unknown.

Provenance

Source

Original MIDI files from an unspecified source, processed by amaai-lab.

Collection Method

Captions produced via a pipeline incorporating MIR feature extraction and the Claude 3 LLM for in-context learning.

Freshness

Last updated on 2025-03-15.