LiveBench is a benchmark designed to limit test set contamination by releasing new questions monthly. Questions are based on recently-released datasets, arXiv papers, news articles, and IMDb movie synopses. Each question has verifiable, objective ground-truth answers, allowing hard questions to be scored.
Use Cases
- Benchmarking LLM performance on mathematical problems based on monthly updates
- Evaluating model resistance to test set contamination using questions from recent sources
- Scoring objective answers on challenging math questions with verifiable ground truth
Strengths
- Questions are released monthly to limit potential contamination
- Questions are sourced from recently-released datasets, arXiv papers, news articles, and IMDb movie synopses
- Each question has verifiable, objective ground-truth answers
Limitations
- Column-level documentation is absent; field semantics must be inferred after download
- Row count is unknown, which may limit suitability assessment
Provenance
- Source
- livebench
- Freshness
- Last updated 2025-04-07 20:34:11; freshness should be verified