AI Paper Atlas

Research map / Large language models / Language, safety and interpretability

Emergent Misalignment is Easy, Narrow Misalignment is Hard

Anna Soligo, Edward Turner, Senthooran Rajamanoharan, Neel Nanda

ICLR 2026 · Large language models › Language, safety and interpretability · approach: Pruning and circuits in LLMs · 14 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
mix single-domain misaligned data with aligned data from other domains (ablation)LoRA finetune on misaligned advice · Emergent misalignmentgeneral and narrow misalignmentNo clear effect
add KL regularisation on out-of-domain data to the finetuning loss (ablation)steering vector or LoRA finetune · Bad Medical Advicegeneral misalignmentImproved
train narrow solution instead of general solution (efficiency: loss at equal parameter norm) (ablation)steering vector · Bad Medical Advicetraining loss at equal normHurt
train narrow solution instead of general solution (stability to orthogonal noise) (ablation)steering vector · Bad Medical Adviceloss increase under noiseHurt
remove KL regularisation and continue finetuning narrow solution (ablation)narrow steering vector · Bad Medical Advicemisalignment generalityHurt
steer with general instead of narrow or random vectors (ablation)misalignment vectors · FineWebKL divergence from chat modelImproved
use text datasets instead of insecure code for finetuningQwen2.5-14B-Instruct · Emergent misalignmentEM and coherency: 6% EM and 67% coherency -> nearing 40% misalignment with over 99% coherencyImproved
use full SFT instead of LoRA (ablation)Qwen2.5-14B-Instruct · Emergent misalignmentEM (%): 8.92 to 36.34No clear effect
use full SFT on Gemma instead of Qwengemma-3-12b-it · Emergent misalignmentEM (%): 3.82 to 13.78 after 3 epochsHurt
use rank-1 LoRA instead of rank-32 LoRA (ablation)Qwen2.5-14B-Instruct · Emergent misalignmentEM (%): 9.25 to 21.5No clear effect
increase model size (ablation)Qwen and Llama models · Emergent misalignmentEM and coherency: up to 8% and 9% EM at smallest sizesImproved
increase model size (ablation)Gemma-3 models · Emergent misalignmentEMNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy