Research map / Large language models / Language, safety and interpretability
To Each (Textual Sequence) Its Own: Improving Memorized-Data Unlearning in Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes per-sequence memorization-aware unlearning for LLMs (Selective Gradient Ascent and Task Arithmetic for Unlearning), a memorization-outlier metric, and a loss-based MIA showing GA leaves forget-set sequences vulnerable.
Problem
Approximate unlearning of memorized training sequences from pretrained LLMs is evaluated with average memorization scores, which hide under- and over-memorized outliers that remain extractable or harm utility.
Approach
Treat each forget-set sequence according to its own exact-memorization score: SGA applies gradient ascent only to sequences above a memorization upper limit (or top-k most memorized), and TAU applies task-arithmetic drift after an SGA warm-up; success is measured by counting per-sequence outliers outside a [15%, 60%] band.
Key results
- On GPT-Neo 125M with 16 forget samples, TAU and TA reduce dialogue F1 by roughly 30% (vs ~80% loss for GA/SGA), while SGA yields the fewest extractable outliers (1.8 at 125M, 1.0 at 1.3B).
- For GPT-Neo 1.3B with 16 samples, SGA achieves F1 12.63 and TAU 12.52 versus GA 11.34 and default 12.86, with 1.0 extractable outlier for SGA versus 4.6 for GA.
- The MIAU attack raises true positive rate at 0% FPR from about 60% to 90% after gradient-ascent unlearning on GPT-Neo 1.3B.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use selective gradient ascent instead of gradient ascent | GPT-Neo · Pile forget set (16 samples), 125M, 1.3B, and 2.7B | EXTRACTABLE: 5.0 -> 1.8; 4.6 -> 1.0; 7.0 -> 0.8 | Helped |
| Use selective gradient ascent instead of gradient ascent | GPT-Neo · Pile forget set (16 samples), 125M, 1.3B, and 2.7B | F1: 2.28 -> 1.86; 11.34 -> 12.63; 10.99 -> 10.94 | Mixed |
| Use task arithmetic instead of gradient ascent | GPT-Neo · Pile forget set (16 samples), 125M, 1.3B, and 2.7B | F1: 2.28 -> 7.18; 11.34 -> 9.60; 10.99 -> 10.03 | Helped |
| Use task arithmetic instead of gradient ascent | GPT-Neo · Pile forget set (16 samples), 125M, 1.3B, and 2.7B | EXTRACTABLE: 5.0 -> 2.2; 4.6 -> 2.0; 7.0 -> 1.6 | Mixed |
| Use task arithmetic for unlearning instead of gradient ascent | GPT-Neo · Pile forget set (16 samples), 125M, 1.3B, and 2.7B | F1: 2.28 -> 7.37; 11.34 -> 12.52; 10.99 -> 11.34 | Mixed |
| Use task arithmetic for unlearning instead of gradient ascent | GPT-Neo · Pile forget set (16 samples), 125M, 1.3B, and 2.7B | EXTRACTABLE: 5.0 -> 3.4; 4.6 -> 4.0; 7.0 -> 4.0 | Mixed |
| Add LoRA to gradient ascent (ablation) | GA on GPT-Neo 125M, 32 samples · Pile forget set | F1: 2.85 -> 7.39 | Helped |
| Add LoRA to gradient ascent (ablation) | GA on GPT-Neo 125M, 32 samples · Pile forget set | EXTRACTABLE: 9.0 -> 7.0 | Helped |
| Add LoRA to selective gradient ascent (ablation) | SGA on GPT-Neo 125M, 32 samples · Pile forget set | F1: 4.64 -> 8.56 | Helped |
| Add LoRA to selective gradient ascent (ablation) | SGA on GPT-Neo 125M, 32 samples · Pile forget set | EXTRACTABLE: 3.2 -> 5.0 | Hurt |
| Add LoRA to task arithmetic (ablation) | TA on GPT-Neo 125M, 32 samples · Pile forget set | F1: 5.76 -> 4.68 | Hurt |
| Add LoRA to task arithmetic (ablation) | TA on GPT-Neo 125M, 32 samples · Pile forget set | EXTRACTABLE: 5.0 -> 5.0 | No clear effect |
Surprising findings
- Task arithmetic performs best for small models (125M) but fails to scale beyond 1B parameters, where it loses utility.
- GA+Mismatch yields fewer extractable outliers than GA but is worse on utility metrics across the board.
Limitations
- No single method dominates across all forget-set and model sizes.
- Privacy is measured only as verbatim memorization; factual associations and PII linkage are not addressed.
- Hyperparameters such as the outlier band [15%, 60%] and memorization thresholds are set empirically.
Experimental setup
Datasets: Pile Training Data Extraction Challenge samples Wizard of Wikipedia Wizard of Internet Blended Skill Talk Empathetic Dialogues ARC-Easy ARC-Challenge COPA LAMBADA Winogrande PubmedQA
Benchmarks: Wizard of Wikipedia Wizard of Internet Blended Skill Talk Empathetic Dialogues ARC-Easy ARC-Challenge COPA LAMBADA Winogrande PubmedQA PIQA HellaSwag
Metrics: F1 cross-entropy accuracy EXTRACTABLE outlier count Delta MEM runtime
Baselines: GA TA DPD GA+Mismatch DEFAULT (pre-unlearning model)
Builds on: gradient ascent unlearning (Yao et al.) task arithmetic differential privacy decoding LoRA
Closest papers by meaning
- A Closer Look at Machine Unlearning for Large Language Models (ICLR 2025 · similarity 0.93)
- MUSE: Machine Unlearning Six-Way Evaluation for Language Models (ICLR 2025 · similarity 0.92)
- Reversing the Forget-Retain Objectives: An Efficient LLM Unlearning Framework from Logit Difference (NeurIPS 2024 · similarity 0.91)
- Towards Effective Evaluations and Comparisons for LLM Unlearning Methods (ICLR 2025 · similarity 0.91)
- Textual Unlearning Gives a False Sense of Unlearning (ICML 2025 · similarity 0.91)
- Unlearning or Obfuscating? Jogging the Memory of Unlearned LLMs via Benign Relearning (ICLR 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.