100 curated instruction-response pairs form a concise evaluation benchmark for large language models. This subset, created by tinyBenchmarks, replicates the core of the AlpacaEval 2.0 benchmark for faster testing. The dataset was last updated in April 2024.
Use Cases
- Benchmark model instruction-following quality by comparing generated outputs against the provided gpt4_turbo reference responses.
- Measure response coherence and relevance using the curated human-written instructions as prompts.
- Conduct rapid, low-cost performance comparisons between different LLMs using the fixed set of 100 evaluation points.
Strengths
- Contains 100 specifically curated data points for efficient testing.
- Includes benchmark outputs from a high-performing model (gpt4_turbo) for comparison.
Limitations
- Small scale of 100 examples may not capture performance on a wide range of tasks.
- As a subset, it may not represent the full statistical distribution of the original 805-example benchmark.
Provenance
- Source
- Subset of the AlpacaEval 2.0 benchmark, hosted by tinyBenchmarks on Hugging Face.
- Collection Method
- Curated selection of 100 examples from the original compilation of 805.
- Time Range
- null
- Freshness
- Last updated in April 2024.
- Geography
- null