TTS Human Preferences: 15,000 Annotations for Audio Quality Evaluation

Datapointai released this dataset in March 2026 containing 1,000 text-to-speech audio pairs and 15,000 human preference annotations. Each entry consists of a single text prompt rendered by two different TTS systems, with 15 human labels indicating which version sounds more natural.

Use Cases

• Training Direct Preference Optimization (DPO) models using the preference labels to align TTS output with human naturalness
• Evaluating TTS model performance by comparing the 15 human annotations against automated quality metrics
• Developing reward models for Reinforcement Learning from Human Feedback (RLHF) in audio synthesis tasks

Strengths

• 15 human annotations per row
• 15,000 total annotations
• CC BY 4.0 license

Limitations

• Small sample size of 1,000 unique prompts
• Limited to English language
• Focuses exclusively on naturalness rather than prosody or emotion

Provenance

Source

datapointai

Collection Method

Human annotation of text-to-speech audio renderings

Freshness

Last updated March 2026.

Geography

United States