Research map / Large language models / Language, safety and interpretability
Exploring Criteria of Loss Reweighting to Enhance LLM Unlearning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Analyzes loss reweighting for LLM unlearning via two goals (saturation and importance), proposes simplified variants SimSat/SimImp and a combined token-wise method SatImp that outperforms GA-based baselines on TOFU, WMDP, and MUSE.
Problem
Gradient-ascent-based LLM unlearning over-forgets and harms retained utility; loss reweighting helps, but the functions and optimal criteria of existing reweighting strategies are unclear.
Approach
Categorizes reweighting into saturation (upweight not-yet-unlearned tokens) and importance (upweight critical tokens), shows they focus on complementary token likelihood regimes, and proposes SatImp, a token-wise weight p^beta1 * (1-p)^beta2 that emphasizes middle-loss tokens.
Key results
- On TOFU Forget-1% with LLaMA2-7B, SatImp reaches ES Re. 0.8034 and ES Un. 0.0579, versus WGA at 0.7603 and 0.0545.
- On WMDP (forget-only), SatImp achieves WMDP-Bio 0.2470 and WMDP-Cyber 0.2456 but MMLU 0.3197, below NPO's 0.5607 retain score.
- Saturation-based reweighting outperforms importance-based reweighting on retention, and their product combination further improves Phi-1.5 ES Re. from 0.1277 to 0.1711 in the table.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| combine importance and saturation weights multiplicatively (ablation) | TOFU unlearning · TOFU 1% | ES Re.: 0.1655 -> 0.1711; 0.1277 -> 0.1711; 0.4082 -> 0.4282; 0.3863 -> 0.4282 | Helped |
| combine importance and saturation weights multiplicatively (ablation) | TOFU unlearning · TOFU 1% | ES Un.: 0.0451 -> 0.0328; 0.0057 -> 0.0328; 0.0918 -> 0.0649; 0.0565 -> 0.0649 | Mixed |
| use saturation-based rather than importance-based reweighting | LLM unlearning · TOFU 1% | ES Re. | Hurt |
| use importance-based rather than saturation-based reweighting | LLM unlearning · TOFU 1% | ES Un. | Helped |
| increase beta to make the weight distribution more peaked (ablation) | SimSat and SimImp · TOFU | ES Re. | Helped |
| increase beta to make the weight distribution more peaked (ablation) | SimSat and SimImp · TOFU | ES Un. | Hurt |
| use finer-grained rather than coarse-grained reweighting (ablation) | SimSat · TOFU | unlearning efficiency | Helped |
| use finer-grained reweighting (ablation) | SimImp · TOFU | retention performance | Helped |
| use finer-grained reweighting (ablation) | SimImp · TOFU | unlearning performance | Hurt |
| use soft reweighting rather than hard sampling (ablation) | LLM unlearning · TOFU | unlearning efficacy | Helped |
| use token-wise rather than instance-wise weights (ablation) | SimSat · TOFU | unlearning-retention trade-off | Helped |
| use token-wise rather than instance-wise weights (ablation) | SimImp · TOFU | unlearning-retention trade-off | Mixed |
Surprising findings
- Hard sampling (TopK/BottomK) underperforms soft reweighting, and fine-grained token-wise weighting causes over-forgetting at larger forget sets.
- The authors find NPO and SimNPO behave mainly as saturation-based reweighting, contradicting the prior assumption that NPO focuses on key tokens.
Limitations
- Lacks a theoretical analysis of the reweighting mechanism.
- SatImp is sensitive to hyperparameters beta1 and beta2.
- Token-wise reweighting leads to over-forgetting on TOFU 5% and 10% settings when forget-only regularization is used.
Experimental setup
Datasets: TOFU WMDP MUSE MMLU
Benchmarks: TOFU WMDP MUSE
Metrics: ES score forget quality model utility accuracy VerbMem KnowMem PrivLeak UtilPres
Baselines: GA PO DPO NPO SimNPO WGA RMU
Builds on: gradient ascent WGA NPO SimNPO TOFU
Closest papers by meaning
- Dynamic Loss-Based Sample Reweighting for Improved Large Language Model Pretraining (ICLR 2025 · similarity 0.93)
- Towards Effective Evaluations and Comparisons for LLM Unlearning Methods (ICLR 2025 · similarity 0.92)
- WAGLE: Strategic Weight Attribution for Effective and Modular Unlearning in Large Language Models (NeurIPS 2024 · similarity 0.92)
- Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning (NeurIPS 2025 · similarity 0.91)
- LLM Unlearning via Loss Adjustment with Only Forget Data (ICLR 2025 · similarity 0.91)
- Rethinking LLM Unlearning Objectives: A Gradient Perspective and Go Beyond (ICLR 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.