AI Paper Atlas

Research map / Continual learning / Continual fine-tuning of LLMs

Attention Smoothing Is All You Need For Unlearning

Saleh Zare Zade, Xiangyu Zhou, Sijia Liu, Dongxiao Zhu

ICLR 2026 · Continual learning › Continual fine-tuning of LLMs · approach: Machine unlearning by selective forgetting · 3 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace fixed-target or divergence forgetting with self-distillation from attention-smoothed forget-teacher (ASU_KL)IDK_AP · TOFU forget05Forget Efficacy: 60.88 -> 77.84Improved
ASU_KL vs IDK_AP on utilityIDK_AP · TOFU forget05Model Utility: 75.23 -> 74.18No clear effect
ASU_KL vs NPO_GDNPO_GD · TOFU forget10Model Utility: 56.58 -> 73.27Improved
ASU_KL on real-world unlearningbest baseline utility · Real-world unlearningModel Utility: 52.76 -> 55.76Improved
ASU_KL on MUSE NewsNPO_KL · MUSE NewsVerbMem on forget: 33.0 -> 8.8Improved
ASU_KL on MUSE BooksNPO_KL · MUSE BooksVerbMem on forget: 54.4 -> 5.3Improved
smooth attention only in shallow layers 6-8 instead of all layers (ablation)ASU teacher · TOFU forget05MU / FE: FE 77.84 -> 77.49 (KL)No clear effect
combine ASU with IDK_AP refusal outputs (ablation)IDK_AP · TOFU forget10Forget Efficacy: 61.27 -> 86.77 (KL)Improved
vary attention temperature tau from 2.0 to 2.8 (ablation)ASU · TOFU forget05MU and FENo clear effect
increase attention temperature tau in the forget teacher (ablation)base model · TOFU factual vs function tokensnegative log-likelihoodMixed
apply ASU on continual unlearning sequencesNPO and IDK · TOFU continual forget05/forget10average of MU and FEImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy