Linguistically Challenging English Text-to-Speech Benchmark

Tricky TTS is a benchmark dataset designed to stress-test text-to-speech models on challenging English text. Each row targets a specific failure mode to separate capable systems from weaker ones. The dataset was created by Trelis and last updated in March 2026.

Use Cases

• Evaluate TTS model robustness by measuring the Character Error Rate (CER) from a round-trip ASR pipeline using Whisper on generated audio.
• Benchmark TTS systems on specific failure modes like handling complex typography or unusual linguistic constructs present in the dataset text.
• Compare custom voice models developed with services like Trelis Voice AI against standardized challenging prompts.

Strengths

• Designed specifically to stress-test TTS models on linguistically and typographically challenging text.
• Evaluation methodology includes a defined round-trip ASR CER metric using Whisper.

Limitations

• Unknown sample size and specific column structure limits reproducibility of analyses.
• Content is restricted to challenging English text, limiting applicability for multilingual or general-purpose TTS evaluation.

Provenance

Source

Trelis

Collection Method

Built with Trelis Studio; evaluations run using Trelis Studio.

Freshness

Last updated March 2026.