Research map / Continual learning / Continual fine-tuning of LLMs

Reversing the Forget-Retain Objectives: An Efficient LLM Unlearning Framework from Logit Difference

NeurIPS 2024 · Continual learning › Continual fine-tuning of LLMs · approach: Machine unlearning by selective forgetting · 4 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ULD, an LLM unlearning method that trains a small LoRA-equipped assistant (built from the first 8 layers of the target) with reversed forget/retain objectives and subtracts its logits from the target's logits.

Problem

Conventional LLM unlearning (maximizing forget loss while minimizing retain loss) suffers from degenerated outputs due to the unbounded forget loss and catastrophic forgetting due to under-representative retain data, and is costly to train.

Approach

Instead of unlearning the target directly, train an assistant LLM to remember the forget documents and to be uniform (forget) on retain documents, then derive the unlearned model by subtracting the assistant's logits from the target's logits. The assistant is built from the first K=8 layers of the target plus frozen weights and LoRA adapters, so training is cheaper and bounded.

Key results

• On TOFU-1%, ULD achieves forget quality 0.99 with model utility 0.62 (no loss vs. original), while baselines like NPO sacrifice about 17% utility on average.
• ULD reduces training time by more than 3x relative to NPO, the most efficient baseline with comparable forget performance.
• On HarryPotter, ULD keeps retain performance nearly unchanged while baselines with best forget performance show over 5% accuracy decline (NPO+GD).

Measured effects reported in the paper

Surprising findings

• Adding LoRA to baseline unlearning methods reduces their forget performance, while ULD uses LoRA on the assistant effectively.
• Naively adding augmented data to baselines hurts their model utility (e.g., NPO+KL utility drops from 0.32 to 0.08).

Limitations

• Inference requires running the assistant LLM alongside the target, which may increase latency.
• Forget-data augmentation is important but creating suitable augmentations is challenging across datasets.
• Results depend on GPT-3.5-generated paraphrases and perturbations, which adds data-generation cost and dependence.

Experimental setup

Datasets: TOFU HarryPotter book chunks C4 WikiText

Benchmarks: TOFU HarryPotter

Metrics: forget quality model utility ROUGE-L BLEU zero-shot accuracy perplexity on WikiText

Baselines: GA GA+GD GA+KL DPO NPO NPO+GD NPO+KL Offset-unlearning (concurrent assistant-LLM method)

Builds on: gradient ascent unlearning DPO NPO proxy-tuning / logit arithmetic

Closest papers by meaning

• LLM Unlearning via Loss Adjustment with Only Forget Data (ICLR 2025 · similarity 0.94)
• Towards Effective Evaluations and Comparisons for LLM Unlearning Methods (ICLR 2025 · similarity 0.92)
• SHRED: Retain-Set-Free Unlearning via Self-Distillation with Logit Demotion (NeurIPS 2026 · similarity 0.92)
• Learning What to Forget: Improving LLM Unlearning via Learned Token-Level Importance (NeurIPS 2026 · similarity 0.92)
• Keeping an Eye on LLM Unlearning: The Hidden Risk and Remedy (NeurIPS 2025 · similarity 0.92)
• A Closer Look at Machine Unlearning for Large Language Models (ICLR 2025 · similarity 0.92)