13948 multiple-choice questions spanning 52 disciplines and four difficulty levels. The dataset is structured for evaluating foundation models, with splits for development, validation, and testing. It was created by ceval and last updated on 2025-07-27.
Use Cases
- Benchmarking model performance across 52 diverse disciplines based on the subject splits.
- Conducting few-shot evaluation using the dev set exemplars with explanations.
- Hyperparameter tuning for models using the dedicated val set.
- Final model testing and scoring using the test set.
- Analyzing model difficulty scaling based on the four difficulty levels.
Strengths
- 13948 questions provide a substantial evaluation corpus.
- 52 disciplines offer broad coverage of knowledge domains.
- Structured splits (dev, val, test) facilitate standardized evaluation workflows.
- Dev set includes five exemplars with explanations for few-shot evaluation.
Limitations
- Column-level documentation is absent; field semantics must be inferred after download.
- Row count is unknown, which may limit suitability assessment.
- Data may reflect bias inherent to the specific sources used for question creation.
Provenance
- Source
- ceval
- Freshness
- Last updated 2025-07-27 03:59:42; freshness should be verified.