Research map / Continual learning / Continual fine-tuning of LLMs
Forget to Know, Remember to Use: Context-Aware Unlearning for Large Language Models
Yuefeng Peng, Parnian Afshar, Megan Ganji, Thomas Butler, Amir Houmansadr, Mingxian Wang, Dezhi Hong
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add KL context term | RMU · TOFU 5% forget, Gemma-2B-IT | Contextual QA LLM-Judge: 0.00 -> 0.99 | Improved |
| add KL context term | RMU · TOFU 5% forget, Qwen3-8B | Contextual QA LLM-Judge: 0.05 -> 0.97 | Improved |
| add KL context term | NPO · TOFU 5% forget, Gemma-2B-IT | Contextual QA LLM-Judge: 0.81 -> 0.98 | Improved |
| add KL context term | NPO · TOFU 5% forget, Qwen3-8B | Contextual QA LLM-Judge: 0.84 -> 0.95 | Improved |
| add KL context term | UNDIAL · TOFU 5% forget, Gemma-2B-IT | Contextual QA LLM-Judge: 0.82 -> 0.98 | Improved |
| add KL context term | UNDIAL · TOFU 5% forget, Qwen3-8B | Contextual QA ROUGE-L: 0.59 -> 0.68 | Improved |
| add KL context term | RMU · TOFU 5% forget, Gemma-2B-IT | Direct QA ROUGE-L (forgetting): 0.04 -> 0.13 | No clear effect |
| add KL context term | NPO · TOFU 5% forget, Gemma-2B-IT | Model utility: 0.57 -> 0.57 | No clear effect |
| add KL context term | RMU · PISTOL A_C, Gemma-2B-IT | Contextual QA LLM-Judge: 0.00 -> 0.80 | Improved |
| add KL context term | GradDiff · TOFU 5% forget, Gemma-2B-IT | Contextual QA LLM-Judge: 0.000 -> 0.945 | Improved |
| add KL context term | SimNPO · TOFU 5% forget, Gemma-2B-IT | Contextual QA LLM-Judge: 0.940 -> 0.970 | Improved |
| add KL context term | RMU · TOFU 5% forget, long noisy context, Gemma-2B-IT | Contextual QA LLM-Judge: 0.010 -> 0.605 | Improved |
Closest papers by meaning
- What to Forget in Unlearning? Forget Set Curation for Language Models (NeurIPS 2026 · similarity 0.92)
- Reversing the Forget-Retain Objectives: An Efficient LLM Unlearning Framework from Logit Difference (NeurIPS 2024 · similarity 0.92)
- In-Context Unlearning: Language Models as Few-Shot Unlearners (ICML 2024 · similarity 0.92)
- Wisdom is Knowing What not to Say: Hallucination-Free LLMs Unlearning via Attention Shifting (NeurIPS 2025 · similarity 0.91)
- Keeping an Eye on LLM Unlearning: The Hidden Risk and Remedy (NeurIPS 2025 · similarity 0.91)
- Large Language Model Unlearning (NeurIPS 2024 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
