Arena-T2I-Hard: 310-Prompt Stress Benchmark for Text-to-Image Models

Arena-T2I-Hard is a 310-prompt stress benchmark for evaluating the faithfulness of text-to-image models. It was created by lmarena-ai and last updated in July 2026. The prompts are drawn from real user requests and are pre-decomposed into a dependency-aware DAG of yes/no questions for scoring.

Use Cases

• Benchmark model faithfulness based on long, multi-entity prompts.
• Evaluate spatial reasoning based on prompts containing spatial relations.
• Test attribute binding based on prompts with multiple attributes and stylistic constraints.
• Assess counting accuracy based on prompts with numerical counts.
• Compare model performance against other benchmarks like DPG-Bench and DSG.

Strengths

• Contains 310 specific prompts designed as a stress test.
• Prompts are drawn from real, hard user requests, suggesting practical relevance.
• Includes a structured scoring mechanism with a dependency-aware DAG of yes/no questions.

Limitations

• Description metadata is limited; actual data quality requires manual inspection after download.
• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.

Provenance

Source

lmarena-ai via Hugging Face.

Collection Method

Drawn from real arena user requests.

Freshness

Last updated 2026-07-02 00:12:38; freshness should be verified.