AI Paper Atlas

Research map / Large language models / Instruction and fine-tuning

In-Training Defenses Against Emergent Misalignment in Language Models

David Kaczér, Magnus Jørgenvåg, Clemens Vetter, Esha Afzal, Robin Haselhorst, Lucie Flek, Florian Mai

ICML 2026 · Large language models › Instruction and fine-tuning · approach: Safety alignment after fine-tuning · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add KL-divergence regularization toward aligned modelSFT on EM datasets, Qwen2.5-7B · EM Code (general)misaligned rate: 4.01 -> 0.38Improved
add LDIFS feature-space l2 regularizationSFT on EM datasets · EM datasets (general)misaligned rate: 25.29 -> 26.75 (Legal)No clear effect
add preventive persona vector steeringSFT on EM datasets, Qwen2.5-7B · EM Legal (general)misaligned rate: 25.29 -> 1.00Improved
add inoculation promptingSFT on EM datasets, Qwen2.5-7B · EM Legal (general)misaligned rate: 25.29 -> 16.46Mixed
interleave 5% automatically selected safety data (Interleaving++)SFT on EM datasets, Qwen2.5-7B · EM Legal (general)misaligned rate: 25.29 -> 0.79Improved
select interleaving data by perplexity gap and filter refusals (vs random) (ablation)Interleaving · EM general domainincoherence vs dataset sizeImproved
add KL-divergence regularizationSFT on OpSwap · OpSwap Tier 1exact match: 30.00 -> 0.00Hurt
add KL-divergence regularizationSFT on FoQA · FoQAexact match: 41.60 -> 44.55Improved
add persona vector steeringGRPO on GSM8K, Qwen2.5-3B · GSM8Kaccuracy: 90.66 -> 38.33Hurt
add 20% interleaved SFT dataGRPO on GSM8K, Qwen2.5-3B · GSM8Kaccuracy: 90.66 -> 93.66Improved
add Persona Vectors or KL to reduce general misalignmentEM fine-tuning · EM Medical in-domainin-domain misalignment: 51.73 -> 29.67 (Persona Vectors)Hurt
add Interleaving++ to preserve narrow misalignmentEM fine-tuning · EM in-domainin-domain misalignment: 51.60 -> 52.80 (Code)No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy