WMDP: Weapons of Mass Destruction Proxy Benchmark for AI Safety

The Weapons of Mass Destruction Proxy (WMDP) benchmark is a dataset of multiple-choice questions serving as a proxy for hazardous knowledge in biosecurity, cybersecurity, and chemical security. It was created by author 'cais' and last updated on the Hugging Face platform on 2024-04-27. The dataset is designed to evaluate hazardous knowledge in large language models and benchmark methods for unlearning such knowledge.

Use Cases

• Evaluating hazardous knowledge in LLMs based on multiple-choice questions in biosecurity, cybersecurity, and chemical security.
• Benchmarking unlearning methods designed to remove hazardous knowledge from AI models.
• Researching AI safety and model alignment concerning weapons of mass destruction-related information.

Strengths

• Dataset is explicitly designed for a dual purpose: evaluation and unlearning benchmarking.
• Focuses on three specific security domains: biosecurity, cybersecurity, and chemical security.
• Last updated on 2024-04-27, indicating recent maintenance.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• Data may reflect bias inherent to the specific construction of the multiple-choice questions.

Provenance

Source

cais on Hugging Face

Collection Method

Likely constructed by domain experts for the purpose of AI safety evaluation.

Freshness

Last updated 2024-04-27 05:37:45