Humanity's Last Exam: 3,000 Multi-Subject Questions for AI Benchmarking

3,000 questions across dozens of subjects including mathematics, humanities, and natural sciences form this multi-modal benchmark. It was developed globally by subject-matter experts for the Center for AI Safety and Scale AI. The dataset page was last updated on 2025-02-18.

Use Cases

• Benchmarking AI model performance on multi-subject knowledge based on the 3,000-question structure
• Evaluating reasoning capabilities in mathematics, humanities, and natural sciences as described
• Assessing the breadth of AI knowledge across dozens of academic subjects mentioned in the description

Strengths

• 3,000 questions provide a substantial testbed for evaluation
• Broad subject coverage includes mathematics, humanities, and natural sciences
• Developed by subject-matter experts, suggesting domain expertise

Limitations

• Column-level documentation is absent; field semantics must be inferred after download
• Row count is unknown, which may limit suitability assessment
• Description metadata is limited; actual data quality requires manual inspection after download

Provenance

Source

Center for AI Safety & Scale AI

Collection Method

Developed globally by subject-matter experts

Freshness

Last updated 2025-02-18 17:20:09; freshness should be verified