HLE-Verified: 1,000-10,000 Revised High-Difficulty Reasoning Items

HLE-Verified is a refined benchmark containing between 1,000 and 10,000 high-difficulty reasoning items across scientific and technical domains. Released by skylenage in early 2026, this version provides a systematic revision of the original Humanity’s Last Exam (HLE) to address community-reported reliability issues.

Use Cases

• Benchmarking LLM performance on high-difficulty scientific reasoning items
• Evaluating model accuracy across technical domains using verified ground-truth solutions
• Analyzing model failure modes on complex, multi-step reasoning tasks

Strengths

• Contains 1,000 to 10,000 verified records
• Systematic revision process based on community feedback
• Multi-domain coverage of scientific and technical subjects

Limitations

• Specific column schema and metadata fields are not explicitly detailed in the source
• Extreme difficulty level may lead to floor effects for smaller or less capable models

Provenance

Source

Arxiv 2602.13964

Collection Method

Systematic verification and revision

Freshness

Updated February 2026