UXBench: Benchmarking LLMs on User Experience from 70K+ Conversations

70,000+ thumbs-up and thumbs-down conversations from Tencent Yuanbao form the first LLM benchmark grounded in real user satisfaction signals. It evaluates models across three user experience tasks covering 8 interaction scenarios and 83 domains. The dataset was created by mengze-hong and last updated on 2026-06-09.

Use Cases

• Benchmarking LLM performance on user satisfaction based on real feedback signals.
• Analyzing failure modes in AI assistant conversations across 10 defined dimensions.
• Training or fine-tuning models to improve user experience in specific interaction scenarios.

Strengths

• Built from over 70,000 real user feedback signals (thumbs-up/down).
• Covers 8 interaction scenarios and 83 distinct domains for broad evaluation.
• Includes 1,000 disliked conversations annotated with 10 failure dimensions in one split.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count for the full dataset is unknown, which may limit suitability assessment.

Provenance

Source

Tencent Yuanbao.

Collection Method

Collected from user thumbs-up and thumbs-down feedback on conversations.

Freshness

Last updated 2026-06-09 11:52:16; freshness should be verified.