Research map / Large language models / Language, safety and interpretability
Beyond Superficial Forgetting: Thorough Unlearning Through Knowledge Density Estimation and Block Re-Insertion
Feng Guo, Yuntao Wen, Shen Gao, Junshuo Zhang, Shuo Shang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add block re-insertion into original model (ablation) | KUnBR · Random Birthdays | RTT forget accuracy (%): 80.2 -> 43.9 | Improved |
| add block re-insertion into original model (ablation) | KUnBR · MMLU | RTT forget accuracy (%): 57.6 -> 28.0 | Improved |
| add block re-insertion into original model (ablation) | KUnBR · Years | RTT forget accuracy (%): 68.3 -> 36.0 | Improved |
| add gradient-difference pre-unlearning warm-up (ablation) | KUnBR · Random Birthdays | RTT forget accuracy (%): 54.1 -> 43.9 | Improved |
| add gradient-difference pre-unlearning warm-up (ablation) | KUnBR · Years | RTT forget accuracy (%): 36.7 -> 36.0 | No clear effect |
| select blocks by knowledge density instead of head, bottom or uniform (ablation) | KUnBR · Years | forget accuracy over epochs | Improved |
| use fewer blocks (M=4) instead of M=8 (ablation) | KUnBR · Years | RTT forget accuracy (%) at Top-K=50%: 30.4 -> 45.6 | Hurt |
| use too many blocks (M=32) instead of M=8 (ablation) | KUnBR · Years | RTT forget accuracy (%) at Top-K=75%: 15.2 -> 29.1 | Hurt |
| KUnBR vs best baseline on RTT robustness | KUnBR · WMDP-Deduped (LLaMA3-8B) | RTT forget accuracy (%): 52.2 -> 38.8 | Improved |
| KUnBR vs NPO and RIA on general capability | KUnBR · MMLU | reasoning (Rea.): 0.00 -> 41.1 | Improved |
| KUnBR on Zephyr-7B recovery | KUnBR · Random Birthdays (Zephyr-7B-beta) | recovery accuracy (%): 7.64 -> 20.4 | Hurt |
| KUnBR training time vs GD | KUnBR · Years | minutes: 20 -> 17 | Improved |
Closest papers by meaning
- SHRED: Retain-Set-Free Unlearning via Self-Distillation with Logit Demotion (NeurIPS 2026 · similarity 0.91)
- RWKU: Benchmarking Real-World Knowledge Unlearning for Large Language Models (NeurIPS 2024 · similarity 0.90)
- Wisdom is Knowing What not to Say: Hallucination-Free LLMs Unlearning via Attention Shifting (NeurIPS 2025 · similarity 0.90)
- Unlearning or Obfuscating? Jogging the Memory of Unlearned LLMs via Benign Relearning (ICLR 2025 · similarity 0.90)
- Reversing the Forget-Retain Objectives: An Efficient LLM Unlearning Framework from Logit Difference (NeurIPS 2024 · similarity 0.90)
- Towards Robust and Parameter-Efficient Knowledge Unlearning for LLMs (ICLR 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
