LiveBench Instruction Following Benchmark

LiveBench is a benchmark for large language models designed to limit test set contamination by releasing new questions monthly. Questions are based on recently-released datasets, arXiv papers, news articles, and IMDb movie synopses. It was created by 'livebench' and last updated in April 2025.

Use Cases

• Evaluating an LLM's ability to follow instructions based on questions derived from recent arXiv papers.
• Benchmarking model performance on objective, verifiable ground-truth answers for questions sourced from news articles.
• Assessing instruction-following capabilities using questions generated from recently-released datasets.
• Testing model responses to prompts based on IMDb movie synopses.

Strengths

• Questions are released monthly to limit test set contamination.
• Each question has verifiable, objective ground-truth answers.

Limitations

• The specific number of questions, rows, and columns is not provided.
• The dataset's size, file formats, and license are unknown.

Provenance

Source

livebench on Hugging Face

Collection Method

Questions are gathered from recently-released datasets, arXiv papers, news articles, and IMDb movie synopses.

Time Range

Ongoing, with monthly updates.

Freshness

Updated monthly with new questions.