WMDP: 1,000+ Hazardous Knowledge Questions with Domain-Specific Splits

WMDP contains between 1,000 and 10,000 multiple-choice questions across biology, chemistry, and cybersecurity domains for evaluating hazardous knowledge in language models. Created by Joschka and based on Arxiv paper 2403.03218, it includes domain-specific configurations and pre-computed splits. The dataset is licensed under MIT and was last updated in March 2026.

Use Cases

• Evaluating model safety by testing knowledge in the wmdp-bio configuration
• Measuring unlearning efficacy using the Lock and Elicit splits
• Benchmarking cybersecurity capabilities through the wmdp-cyber subset

Strengths

• 1,000 to 10,000 domain-specific records
• Pre-computed Lock/Elicit/Test splits for standardized evaluation
• MIT licensed for open research use
• Based on peer-reviewed research (Arxiv 2403.03218)

Limitations

• Small sample size of under 10,000 total records
• Limited to three specific scientific and technical domains
• Proxy-based nature may not capture all nuances of real-world WMD risks

Provenance

Source

Arxiv paper 2403.03218

Freshness

Last updated March 2026