Research map / Federated learning and privacy / Privacy attacks and secure learning

Textual Unlearning Gives a False Sense of Unlearning

ICML 2025 · Federated learning and privacy › Privacy attacks and secure learning · approach: Membership inference and auditing · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that textual unlearning in language models leaves unlearned texts detectable via a rigorous auditing method (U-LiRA+) and creates new leakage risks, exploited by membership inference (TULA-MI) and gradient-based data reconstruction (TULA-DR) attacks.

Problem

Whether textual unlearning in language models truly erases specific training texts, and whether comparing models before and after unlearning leaks new private information about the unlearned texts.

Approach

Audits unlearning with mislabeled "zero-memory" samples injected into training so the model has never seen them before training (worst-case auditing via per-sample likelihood ratio tests), and attacks by exploiting weight differences before and after unlearning, both for membership inference (black/relaxed box) and for reconstructing unlearned text by matching gradients to the weight difference.

Key results

• On OPT-1.3b with KL unlearning, the baseline U-LiRA reports TPR falling from 12.25% to 0.28%, while U-LiRA+ shows TPR still at 41.60% after unlearning.
• TULA-MI in the relaxed black-box setting accurately infers membership for over 18% of samples at a wrong-guess rate below 0.1%.
• TULA-DR reconstructs over 80% of original words (Pythia-1.4b, R-1 up to 81-93% depending on method) for unlearned texts from weight differences.

Measured effects reported in the paper

Surprising findings

• Retrain (exact unlearning) is more vulnerable to the strict black-box membership inference than inexact methods in the reported tables.
• Existing per-sample and population auditing methods substantially overestimate erasure effectiveness of inexact unlearning.

Limitations

• Evaluation restricted to two synthetic binary text classification datasets and two models of ~1.3-1.4B parameters
• Attacks in white-box setting assume the adversary knows the unlearning algorithm and hyperparameters
• Mislabeled audit samples represent a worst-case scenario and may not reflect typical unlearning requests

Experimental setup

Datasets: SynthPAI-age SynthPAI-inc

Benchmarks: SynthPAI-age SynthPAI-inc

Metrics: TPR@0.1%FPR NTS@1FS ROUGE-1 ROUGE-2 ROUGE-L AUC-ROC

Baselines: U-LiRA LiRA ULeaks Retrain GA KL NPO TaskVec

Builds on: LiRA Negative Preference Optimization Task vectors ULeaks

Closest papers by meaning

• Keeping an Eye on LLM Unlearning: The Hidden Risk and Remedy (NeurIPS 2025 · similarity 0.93)
• Unlearned but Not Forgotten: Data Extraction after Exact Unlearning in LLM (NeurIPS 2025 · similarity 0.92)
• Unlearning or Obfuscating? Jogging the Memory of Unlearned LLMs via Benign Relearning (ICLR 2025 · similarity 0.92)
• To Each (Textual Sequence) Its Own: Improving Memorized-Data Unlearning in Large Language Models (ICML 2024 · similarity 0.91)
• MUSE: Machine Unlearning Six-Way Evaluation for Language Models (ICLR 2025 · similarity 0.91)
• A Closer Look at Machine Unlearning for Large Language Models (ICLR 2025 · similarity 0.91)