LiveBench is a benchmark for large language models (LLMs) designed with test set contamination and objective evaluation in mind. The dataset, created by 'livebench', includes questions based on recently-released datasets, arXiv papers, news articles, and IMDb movie synopses. It was last updated on 2025-04-07.
Use Cases
- Benchmarking LLM performance on objective reasoning tasks based on the monthly question release schedule.
- Evaluating model generalization on questions derived from recent datasets and arXiv papers as described.
- Scoring model answers against verifiable ground-truth answers mentioned in the description.
- Assessing LLM capabilities on questions sourced from recent news articles and IMDb movie synopses.
Strengths
- Designed to limit potential test set contamination by releasing new questions monthly.
- Questions are based on recently-released datasets, arXiv papers, news articles, and IMDb movie synopses.
- Each question has verifiable, objective ground-truth answers for scoring.
Limitations
- Column-level documentation is absent; field semantics must be inferred after download.
- Row count, file formats, and license information are unknown, which may limit suitability assessment.
Provenance
- Source
- livebench
- Collection Method
- Questions are compiled from recently-released datasets, arXiv papers, news articles, and IMDb movie synopses.
- Time Range
- Questions are released monthly.
- Freshness
- Last updated 2025-04-07 20:34:13.
- Geography
- null