Research map / Adversarial robustness and security / Attacks on text and graphs
Privacy Backdoors: Enhancing Membership Inference through Poisoning Pre-trained Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes privacy backdoors: poisoning a public pre-trained model (CLIP or LLM) so that a victim's later fine-tuning leaks membership of target data points at much higher rates in black-box membership inference.
Problem
Fine-tuning from publicly distributed pre-trained checkpoints leaves victims' fine-tuning data vulnerable to membership inference; the question is whether an adversary can amplify this leakage by poisoning the released pre-trained model alone, without controlling the victim's data or training.
Approach
Poison the pre-trained weights with an objective that maximizes loss on target points (for CLIP) or minimizes loss on target points (for LLMs/encoders), plus an auxiliary clean-data loss to preserve utility, so that after the victim fine-tunes, members and non-members have strongly differing losses that a LiRA or loss-based attack can exploit.
Key results
- On CLIP ImageNet, poisoning raises TPR@1%FPR from 0.188 to 0.503 and AUC from 0.744 to 0.932 for the membership inference attack.
- On LLM canary experiments, the minimization poisoning raises TPR@1%FPR on the simple PII dataset from 0.242 to 0.963 and on MIMIC-IV from 0.024 to 0.933 with GPT-Neo-125M/ClinicalBERT-style setups.
- Under Neftune fine-tuning on ai4Privacy, TPR@1%FPR rises from 0.048 to 0.725; with 8-bit inference, poisoned TPR is 0.696 versus 0.049 without poisoning.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add privacy backdoor poisoning | CLIP ViT-B-32 · CIFAR-10 | TPR@1%FPR: 0.026 -> 0.131 | Helped |
| Add privacy backdoor poisoning | CLIP ViT-B-32 · CIFAR-100 | TPR@1%FPR: 0.059 -> 0.164 | Helped |
| Add privacy backdoor poisoning | CLIP ViT-B-32 · ImageNet | TPR@1%FPR: 0.188 -> 0.503 | Helped |
| Add privacy backdoor poisoning | GPT-Neo-125M · Simple PII | TPR@1%FPR: 0.242 -> 0.963 | Helped |
| Add privacy backdoor poisoning | GPT-Neo-125M · ai4Privacy | TPR@1%FPR: 0.049 -> 0.874 | Helped |
| Add privacy backdoor poisoning | ClinicalBERT · MIMIC-IV | TPR@1%FPR: 0.024 -> 0.933 | Helped |
| Use linear probing instead of full fine-tuning (ablation) | Privacy backdoor attack · ImageNet | TPR@1%FPR: 0.024 -> 0.324 | Helped |
| Use LoRA instead of full fine-tuning (ablation) | Privacy backdoor attack · ai4Privacy | TPR@1%FPR: 0.020 -> 0.326 | Helped |
| Use 4-bit QLoRA instead of full fine-tuning (ablation) | Privacy backdoor attack · ai4Privacy | TPR@1%FPR: 0.016 -> 0.049 | Helped |
| Use 8-bit QLoRA instead of full fine-tuning (ablation) | Privacy backdoor attack · ai4Privacy | TPR@1%FPR: 0.018 -> 0.065 | Helped |
| Use Neftune instead of standard fine-tuning (ablation) | Privacy backdoor attack · ai4Privacy | TPR@1%FPR: 0.048 -> 0.725 | Helped |
| Restrict inference to top-5 probabilities (ablation) | Privacy backdoor attack · ai4Privacy | TPR@1%FPR: 0.028 -> 0.448 | Helped |
Surprising findings
- Poisoned targets also increase leakage of non-target data points from the same distribution (TPR@1%FPR 0.664% on ai4Privacy vs 0.049% baseline).
- The privacy backdoor persists with up to 20,000 fine-tuning steps, though its effectiveness slightly decreases with fewer steps.
Limitations
- Attack effectiveness drops for LoRA and QLoRA fine-tuning and with 4-bit quantization, and the backdoor weakens with fewer fine-tuning steps.
- Threat model assumes the adversary knows target data points in advance (targeted setting) and that the victim fine-tunes the poisoned checkpoint.
- Poisoning maximization slightly reduces zero-shot accuracy of poisoned CLIP models, which may make the attack detectable.
Experimental setup
Datasets: ImageNet CIFAR-10 CIFAR-100 WikiText-103 ai4Privacy MIMIC-IV MIMIC-III
Benchmarks: ImageNet CIFAR-10 CIFAR-100 WikiText-103 ai4Privacy MIMIC-IV
Metrics: TPR@1%FPR AUC accuracy validation loss
Baselines: No Poison (benign pre-trained model) LiRA Neftune fine-tuning baseline
Builds on: LiRA Carlini et al. canary extraction / exposure neurotoxin-style targeted poisoning (mislabeled-data MIA) LoRA NEFTune
Closest papers by meaning
- Forget to Flourish: Leveraging Machine-Unlearning on Pretrained Language Models for Privacy Leakage (AAAI 2025 · similarity 0.92)
- Privacy Backdoors: Stealing Data with Corrupted Pretrained Models (ICML 2024 · similarity 0.91)
- Backdoor Secrets Unveiled: Identifying Backdoor Data with Optimized Scaled Prediction Consistency (ICLR 2024 · similarity 0.88)
- Demystifying Poisoning Backdoor Attacks from a Statistical Perspective (ICLR 2024 · similarity 0.88)
- Lie Detector: Unified Backdoor Detection via Cross-Examination Framework (NeurIPS 2025 · similarity 0.88)
- IBAS:Imperceptible Backdoor Attacks in Split Learning with Limited Information (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.