Research map / Large language models / Instruction and fine-tuning
Deep Ignorance: Filtering Pretraining Data Builds Tamper-Resistant Safeguards into Open-Weight LLMs
Kyle O'Brien, Stephen Casper, Quentin Gregory Anthony, Tomek Korbak, Robert Kirk, Xander Davies, Ishan Mishra, Geoffrey Irving, Yarin Gal, Stella Biderman
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| filter biothreat-proxy documents from pretraining data (strong filter) | 6.9B LLM · WMDP-Bio Verified Cloze | accuracy: 36.34 -> 24.44 | Improved |
| filter biothreat-proxy documents from pretraining data (strong filter) | 6.9B LLM · WMDP-Bio Robust MCQA | accuracy: 42.97 -> 35.37 | Improved |
| data filtering vs Circuit Breaking on MCQA | 6.9B LLM · WMDP-Bio Robust MCQA | accuracy | Hurt |
| strong filtering on general capabilities | 6.9B LLM · MMLU, PIQA, LAMBADA, HellaSwag average | accuracy: 56.05 -> 55.53 | No clear effect |
| add multi-stage ModernBERT review to annealing filter (ablation) | filtered 6.9B LLM · general knowledge average | accuracy: 55.53 -> 57.88 | Improved |
| single-stage (blocklist only) instead of multi-stage filtering (ablation) | filtered 6.9B LLM · WMDP-Bio average proxy knowledge | accuracy: 30.50 -> 29.90 | Improved |
| pretraining data filtering against adversarial fine-tuning up to 10,000 steps and 305M tokens | 6.9B LLM · WMDP-Bio | accuracy | Improved |
| pretraining data filtering under benign WikiText fine-tuning | 6.9B LLM · WMDP-Bio | accuracy | Improved |
| pretraining data filtering against latent-space attacks | 6.9B LLM · WMDP-Bio MCQA vs cloze | accuracy | Mixed |
| pretraining data filtering against in-context retrieval (open-book) | 6.9B LLM · 1,000-question open-book biothreat MCQA | accuracy | No clear effect |
| add Circuit Breaking to filtered model (ablation) | filtered 6.9B LLM · open-book biothreat MCQA | accuracy | Improved |
| add Circuit Breaking to filtered model (ablation) | filtered 6.9B LLM · fewshot and latent-space attacks | accuracy | Improved |
Closest papers by meaning
- Safeguarding LLMs via Model-Agnostic Latent Safety Signals from Dark Knowledge (NeurIPS 2026 · similarity 0.88)
- Breach By A Thousand Leaks: Unsafe Information Leakage in 'Safe' AI Responses (ICLR 2025 · similarity 0.88)
- Scaling Trends for Data Poisoning in LLMs (AAAI 2025 · similarity 0.88)
- Bits Leaked per Query: Information-Theoretic Bounds for Adversarial Attacks on LLMs (NeurIPS 2025 · similarity 0.87)
- Soft Prompt Threats: Attacking Safety Alignment and Unlearning in Open-Source LLMs through the Embedding Space (NeurIPS 2024 · similarity 0.87)
- Learning and Forgetting Unsafe Examples in Large Language Models (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
