Research map / Federated learning and privacy / Privacy attacks and secure learning

Forget to Flourish: Leveraging Machine-Unlearning on Pretrained Language Models for Privacy Leakage

AAAI 2025 · Federated learning and privacy › Privacy attacks and secure learning · approach: Membership inference and auditing · 5 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes bounded unlearning on noisy copies of private fine-tuning data as a poisoning step in a pre-trained LLM, which amplifies membership inference and data extraction leakage after victim fine-tuning while keeping validation perplexity nearly unchanged.

Problem

Pre-trained models distributed on community hubs can be poisoned to increase privacy leakage of data used in later fine-tuning, with the attacker having only black-box access to the fine-tuned model.

Approach

Gradient-ascent unlearning on noisy versions of the target data, constrained so that loss on a plain-text utility set stays below a threshold, makes the victim's fine-tuning overfit members more strongly than non-members, amplifying MIA and DEA success.

Key results

• On Llama2-7B full fine-tuning with MIND, reference model-based poisoned attacks improve AUC by about 7.5% and Best Acc by about 7% over baseline.
• On Wiki+PII with Llama2-7B full fine-tuning, Poison-word-Rel reaches about 62% TPR at 1% FPR versus 19% for the baseline reference attack.
• Data extraction with DEA-Gen on Llama2-7B full fine-tuning on MIND yields 177 successful reconstructions out of 500 versus 93 for baseline.

Measured effects reported in the paper

Surprising findings

• Differential privacy at epsilon up to 50 significantly degrades utility, making the model almost unusable, so DP offers a poor privacy-utility trade-off against this attack.
• Unlearning on noisy data up to 30% noise level slightly reduced validation perplexity, suggesting utility can improve.

Limitations

• Attack success drops substantially under LoRA and QLoRA fine-tuning, especially on PII data.
• Membership inference and extraction performance is lower on GPT-Neo than Llama2-7B.
• Evaluation limited to two datasets and two models.

Experimental setup

Datasets: MIND WikiText-103 AI4Privacy pii-masking-200k Wiki-2

Benchmarks: MIND Wiki+PII

Metrics: AUC Best Acc TPR at 1% FPR TPR at 0.1% FPR Val-PPL NSR

Baselines: Baseline-loss Baseline-Rel loss-minimization poisoning (concurrent work) DEA-Baseline

Builds on: gradient ascent unlearning DP-transformers LoRA QLoRA

Closest papers by meaning

• Unlearned but Not Forgotten: Data Extraction after Exact Unlearning in LLM (NeurIPS 2025 · similarity 0.92)
• Privacy Backdoors: Enhancing Membership Inference through Poisoning Pre-trained Models (NeurIPS 2024 · similarity 0.92)
• Privacy Auditing of Large Language Models (ICLR 2025 · similarity 0.90)
• Textual Unlearning Gives a False Sense of Unlearning (ICML 2025 · similarity 0.90)
• Underestimated Privacy Risks for Minority Populations in Large Language Model Unlearning (ICML 2025 · similarity 0.90)
• PrivAuditor: Benchmarking Data Protection Vulnerabilities in LLM Adaptation Techniques (NeurIPS 2024 · similarity 0.89)