LiveBench Monthly Questions for LLM Evaluation

LiveBench releases new questions monthly to limit test set contamination for large language models. The benchmark includes questions derived from recently-released datasets, arXiv papers, news articles, and IMDb movie synopses. The dataset was created by 'livebench' and last updated in April 2025.

Use Cases

• Benchmarking LLM performance on questions derived from recent arXiv papers.
• Evaluating model capabilities on objective, verifiable answers from news article content.
• Testing for contamination by using monthly-released questions based on IMDb movie synopses.
• Assessing LLM reasoning on problems constructed from newly-released datasets.

Strengths

• Questions are released monthly to actively limit test set contamination.
• Sources include recently-released datasets and arXiv papers, ensuring topical relevance.
• Each question has verifiable, objective ground-truth answers for reliable scoring.

Limitations

• Specific row count, column details, and sample size are not provided.
• The dataset's internal structure, question formats, and answer keys are not described.
• License information is unknown, which may restrict usage.

Provenance

Source

Hugging Face dataset 'livebench/data_analysis'.

Collection Method

Questions constructed from recently-released datasets, arXiv papers, news articles, and IMDb movie synopses.

Time Range

Ongoing monthly releases.

Freshness

Updated monthly with new questions; last platform update was April 7, 2025.

Geography

null