SongEval: 2,399 Complete Songs with Expert Aesthetic Annotations

SongEval contains 2,399 complete songs totaling approximately 140 hours of audio, released by ASLP-lab in 2025. It serves as a benchmark for the aesthetic evaluation of music generation systems, featuring annotations from 16 expert raters across five perceptual dimensions.

Use Cases

• Benchmarking music generation models using the five perceptual dimension scores
• Training reward models for Reinforcement Learning from Human Feedback (RLHF) in audio synthesis
• Analyzing the correlation between expert aesthetic ratings and automated audio metrics

Strengths

• 2,399 complete songs totaling ~140 hours of audio
• Expert-annotated by 16 qualified raters
• Multi-dimensional evaluation across 5 perceptual categories

Limitations

• Sample size of 2,399 may not represent the full diversity of global musical genres
• Subjective nature of aesthetic dimensions despite expert involvement
• Non-commercial license restricts industrial application

Provenance

Source

ASLP-lab (Arxiv 2505.10793)

Collection Method

Expert annotation of complete audio tracks

Freshness

Last updated August 2025; corresponds to Arxiv publication 2505.10793.