Humanity's Last Exam Biology and Chemistry Gold Standard

Humanity's Last Exam (HLE) Bio/Chem Gold is a curated subset of a challenging question-answering benchmark for advanced academic fields. The dataset contains biology and chemistry questions audited by FutureHouse, which found approximately 30% of original answers were contradicted by peer-reviewed sources. It was created by FutureHouse and last updated in July 2025.

Use Cases

• Training and fine-tuning large language models on expert-level biology and chemistry questions to improve scientific reasoning.
• Benchmarking model performance on the audited 'gold' subset to measure factual accuracy against peer-reviewed sources.
• Analyzing error patterns in model responses by comparing them to the corrected answers provided in this dataset.
• Studying the specific question types and topics within the biology and chemistry subsets where original benchmark answers were found to be incorrect.

Strengths

• Dataset is specifically audited, with around 30% of questions corrected based on peer-reviewed sources.
• Focuses on two challenging, advanced academic domains: biology and chemistry.
• Part of the larger 'Humanity's Last Exam' benchmark, providing context for model evaluation.

Limitations

• Exact size (row count) and specific column schema are unknown from the provided description.
• Limited to the biology and chemistry subsets, not the full HLE benchmark covering math, physics, engineering, and computer science.
• Potential for undisclosed class imbalance or topic bias within the selected subsets.

Provenance

Source

FutureHouse, derived from the Humanity's Last Exam (HLE) benchmark.

Collection Method

Audited using a combination of expert human evaluators and an in-house research agent.

Freshness

Last updated on 2025-07-23.