Research map / Large language models / AI and society
The WMDP Benchmark: Measuring and Reducing Malicious Use with Unlearning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces the WMDP benchmark of 3,668 multiple-choice questions on biosecurity, cybersecurity, and chemical security as a proxy for hazardous LLM knowledge, plus RMU, a representation-misdirection unlearning method that cuts WMDP-Bio/Cyber accuracy to near random while mostly retaining MMLU and MT-Bench.
Problem
Measuring hazardous knowledge (biosecurity, cybersecurity, chemical security) in LLMs is done with private, narrow evaluations, and there is no public benchmark for research on removing such knowledge via unlearning.
Approach
Use a public multiple-choice proxy benchmark for hazardous knowledge and an unlearning method (RMU) that perturbs intermediate-layer activations on hazardous text toward a fixed random vector while regularizing activations on benign text to match the frozen model.
Key results
- RMU on Yi-34B reduces WMDP-Bio from 75.3% to 30.7% and WMDP-Cyber from 49.7% to 29.0% while MMLU drops only from 72.6% to 70.6%.
- RMU on zephyr-7b reduces WMDP-Bio to 31.2% and WMDP-Cyber to 28.2% with MMLU 57.1% (base 58.1%) and MT-Bench 7.10 (base 7.33).
- Linear probes on RMU-unlearned models achieve only slightly better than random accuracy across layers, and GCG fails to jailbreak RMU-unlearned Yi-34B within 2,500 steps versus under 50 steps for the base model.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply RMU unlearning on WMDP-Bio and WMDP-Cyber | zephyr-7b · WMDP-Bio | accuracy: 63.7 -> 31.2 | Helped |
| Apply RMU unlearning on WMDP-Bio and WMDP-Cyber | zephyr-7b · WMDP-Cyber | accuracy: 44.0 -> 28.2 | Helped |
| Apply RMU unlearning on WMDP-Bio and WMDP-Cyber | Yi-34b · WMDP-Bio | accuracy: 75.3 -> 30.7 | Helped |
| Apply RMU unlearning on WMDP-Bio and WMDP-Cyber | Yi-34b · WMDP-Cyber | accuracy: 49.7 -> 29.0 | Helped |
| Apply RMU unlearning on WMDP-Bio and WMDP-Cyber | Mixtral-8x7B · WMDP-Bio | accuracy: 74.8 -> 34.0 | Helped |
| Apply RMU unlearning on WMDP-Bio and WMDP-Cyber | Mixtral-8x7B · WMDP-Cyber | accuracy: 52.0 -> 30.8 | Helped |
| Apply RMU unlearning on WMDP-Bio and WMDP-Cyber | zephyr-7b · MMLU (All) | accuracy: 58.1 -> 57.1 | No clear effect |
| Apply RMU unlearning on WMDP-Bio and WMDP-Cyber | Yi-34b · MT-Bench | score: 7.65 -> 7.59 | No clear effect |
| Apply RMU unlearning on WMDP-Bio and WMDP-Cyber | zephyr-7b · MT-Bench | score: 7.33 -> 7.10 | Hurt |
| Apply RMU unlearning on WMDP-Bio and WMDP-Cyber | zephyr-7b · MMLU Virology | accuracy: 52.4 -> 25.9 | Hurt |
| Apply RMU unlearning on WMDP-Bio and WMDP-Cyber | zephyr-7b · WMDP-Chem | accuracy: 45.8 -> 45.8 | No clear effect |
| Replace CUT's steering direction with a random vector of the same norm (ablation) | CUT / RMU · WMDP-Bio and WMDP-Cyber | accuracy | No clear effect |
Surprising findings
- Finetuning on the cybersecurity forget corpus recovers WMDP-Cyber performance after RMU, so RMU does not prevent relearning in open-weight settings.
- RMU substantially degrades MMLU virology and computer security, indicating imprecise unlearning of closely related fields.
Limitations
- RMU reduces accuracy on closely related fields such as introductory virology and computer security.
- Unlearned knowledge can be recovered by finetuning, so RMU does not address open-weight relearning threats.
- Chemistry unlearning was not performed because benefits versus capability costs were uncertain.
Experimental setup
Datasets: WMDP MMLU MT-Bench Wikitext PubMed papers (forget/retain corpora) GitHub crawl (cyber forget/retain corpora)
Benchmarks: WMDP-Bio WMDP-Cyber WMDP-Chem MMLU MT-Bench GCG jailbreak robustness
Metrics: accuracy (%) MT-Bench score linear probe accuracy
Baselines: LLMU SCRUB SSD GPT-4 Cut
Builds on: representation engineering SCRUB SSD LLMU GCG
Closest papers by meaning
- NYU CTF Bench: A Scalable Open-Source Benchmark Dataset for Evaluating LLMs in Offensive Security (NeurIPS 2024 · similarity 0.87)
- HarmAug: Effective Data Augmentation for Knowledge Distillation of Safety Guard Models (ICLR 2025 · similarity 0.86)
- Poisoning Attacks on LLMs Require a Near-constant Number of Poison Samples (NeurIPS 2026 · similarity 0.85)
- CTIBench: A Benchmark for Evaluating LLMs in Cyber Threat Intelligence (NeurIPS 2024 · similarity 0.85)
- Cybench: A Framework for Evaluating Cybersecurity Capabilities and Risks of Language Models (ICLR 2025 · similarity 0.85)
- Humanizing the Machine: Proxy Attacks to Mislead LLM Detectors (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.