Research map / Large language models / Instruction and fine-tuning

Mark, Don't Erase: Token Inoculation for Dual-Use Knowledge in LLMs

NeurIPS 2026 · Large language models › Instruction and fine-tuning · approach: Safety alignment after fine-tuning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Token Inoculation, which marks hazardous sentences with a special token during continued pre-training and then uses SFT to answer hazardous questions only when the token is present, retaining dual-use knowledge while gating its expression.

Problem

Safety interventions for dual-use knowledge either destroy hazardous content (unlearning, filtering) or suppress it at the output (refusal training), both at the cost of adjacent-domain competence or over-refusal.

Approach

Retain hazardous knowledge but condition its expression on a privileged control token: inoCPT binds the token to hazard-marked domain text via continued pre-training, and inoSFT branches behavior (answer with token, refuse without), with the token only effective in the assistant turn.

Key results

• On Qwen2.5-14B, Token Inoculation reduces WMDP-Bio accuracy from 79.2% to 17.9% while keeping MMLU at 74.2 (vs 79.7 instruct), SU-F1 0.831 vs 0.688 for RMU.
• Inoculation-accuracy do(X) intervention yields Spearman rho=0.54 (p<0.01, n=24) between inoculation accuracy and refusal selectivity.
• On cybersecurity (Qwen2.5-14B), WMDP-Cyber drops from 49.5% to 10.9% while retaining 83% of MMLU-Computer-Security.

Surprising findings

• Injecting the control token into the user turn triggers blanket refusal rather than unlocking, and tokenizer shielding reduces WMDP-Bio to 2.3% on Qwen2.5-14B.
• Llama 3.1-8B over-refuses benign queries under adversarial deployment scenarios, and the best marking threshold differs between MCQ and open-ended formats.

Limitations

• Continued pre-training starts from an already instruction-tuned model rather than from-scratch pre-training; open-ended corpus is rewritten from MCQs, and open-ended effects are attenuated.
• Marking relies on a heuristic embedding-based hazard score that can mark benign dual-use-like passages; evaluated only on biosecurity and cybersecurity.
• Deployment analysis covers only simple token-injection attacks, not adaptive adversaries.

Experimental setup

Datasets: WMDP-Bio forget corpus WikiText-2 UltraChat CommonsenseQA ARC WMDP cyber-retain corpus

Benchmarks: WMDP-Bio MMLU-Virology MMLU-Biology MMLU-NonSTEM WMDP-Cyber MMLU-Computer-Security MMLU-Computer-Science

Metrics: accuracy SU-F1 (safety-utility harmonic mean) refusal selectivity refusal rate Spearman correlation

Baselines: RMU NPO DPO SimNPO GA Vanilla Refusal

Builds on: Inoculation Prompting RMU NPO DPO Synthetic Document Finetuning

Closest papers by meaning

• AlphaAlign: Incentivizing Safety Alignment with Extremely Simplified Reinforcement Learning (ICLR 2026 · similarity 0.87)
• LLMs Encode Harmfulness and Refusal Separately (NeurIPS 2025 · similarity 0.87)
• The Hidden Dimensions of LLM Alignment: A Multi-Dimensional Analysis of Orthogonal Safety Directions (ICML 2025 · similarity 0.87)
• Palette: A Modular, Controllable, and Efficient Framework for On-demand Authorized Safety Alignment Relaxation in LLMs (NeurIPS 2026 · similarity 0.87)
• Safety Reasoning with Guidelines (ICML 2025 · similarity 0.86)
• Improving LLM Safety Alignment with Dual-Objective Optimization (ICML 2025 · similarity 0.86)