Research map / Large language models / Retrieval and knowledge
ALTER: Asymmetric LoRA for Token-Entropy-Guided Unlearning of LLMs
Xunlei Chen, Jinyu Guo, Yuang Li, Zhaokun Wang, Yi Gong, Jie Zou, Jiwei Wei, Wenhong Tian
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace single LoRA with asymmetric LoRA (shared A, separate B experts) (ablation) | LoRA unlearning · WMDP-Bio / MMLU (Llama3-8B) | MMLU accuracy (%): 39.6 -> 55.3 | Improved |
| add token-entropy-guided routing and Tsallis-entropy loss on top of AsymLoRA (ablation) | AsymLoRA · WMDP-Cyber (Llama3-8B) | accuracy (%): 28.8 -> 25.6 | Improved |
| add token-entropy guidance on top of AsymLoRA (ablation) | AsymLoRA · WMDP fluency (Llama3-8B) | Flu-mean: 2.23 -> 3.46 | Improved |
| add token-entropy guidance on top of AsymLoRA (ablation) | AsymLoRA · MMLU after WMDP unlearning (Llama3-8B) | accuracy (%): 55.3 -> 57.8 | Improved |
| ALTER vs gradient ascent / random label baselines | unlearning methods · MMLU after WMDP unlearning (Llama3-8B) | accuracy (%): 24.8 -> 57.8 | Improved |
| ALTER vs RMU | unlearning methods · WMDP-Bio (Llama3-8B) | accuracy (%): 49.4 -> 24.4 | Improved |
| apply ALTER for unlearning Harry Potter (ablation) | Llama-2-7B · Harry Potter (MUSE) | MMLU: 38.9 (LoRA) -> 44.6 | Improved |
| ALTER vs KL minimization on forget metric | unlearning methods · Harry Potter (MUSE) | ASG: 0.8 (KL) vs 1.3 (ALTER) | No clear effect |
| use AsymLoRA framework instead of full-parameter baselines | unlearning · TOFU-10% (Llama2-7B) | training time: time cost reduced by 86.1% to 87.1% | Improved |
| add token-entropy guidance to AsymLoRA (ablation) | AsymLoRA · TOFU-10% (Llama2-7B) | training time: 1.0 -> 1.25x | Hurt |
| use ALTER in sequential unlearning rounds | Llama3-8B on TOFU · TOFU sequential unlearning 1% to 10% | model utility | Improved |
Closest papers by meaning
- Towards Robust and Parameter-Efficient Knowledge Unlearning for LLMs (ICLR 2025 · similarity 0.92)
- Unified Parameter-Efficient Unlearning for LLMs (ICLR 2025 · similarity 0.91)
- Reversing the Forget-Retain Objectives: An Efficient LLM Unlearning Framework from Logit Difference (NeurIPS 2024 · similarity 0.91)
- Wisdom is Knowing What not to Say: Hallucination-Free LLMs Unlearning via Attention Shifting (NeurIPS 2025 · similarity 0.91)
- Constrained Entropic Unlearning: A Primal-Dual Framework for Large Language Models (NeurIPS 2025 · similarity 0.91)
- LLM Unlearning via Loss Adjustment with Only Forget Data (ICLR 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
