Research map / Large language models / Instruction and fine-tuning
PrivAuditor: Benchmarking Data Protection Vulnerabilities in LLM Adaptation Techniques
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Benchmarks membership inference privacy leakage across seven LLM adaptation techniques (full fine-tuning, adapters, LoRA, prompt/prefix/P-tuning, top-layer tuning, in-context learning), five LLM architectures, and three sensitive-domain datasets, releasing the PrivAuditor codebase.
Problem
Privacy risks of adapting pre-trained LLMs on sensitive domain data have been studied only for narrow settings (mostly encoder models, few fine-tuning methods, few attacks), with no comprehensive benchmark of membership inference across adaptation techniques.
Approach
A unified benchmark that evaluates seven membership inference attacks against many adaptation techniques and LLM architectures on sensitive domain datasets, correlating leakage with trainable parameter count, data size, iterations, and model size.
Key results
- Likelihood-ref is the most effective attack overall; on CorpClimate with T5-Large, LoRA reaches AUC 0.724 under likelihood-ref and Full fine-tuning 0.853 vs. prompt-tuning 0.609.
- Attack AUC generally rises with trainable parameter count: on OPT-6.7B CorpClimate, Full fine-tuning gives 0.751 vs. prompt-tuning 0.729 under likelihood-ref.
- Larger models show higher downstream leakage: OPT AUC rises from 0.699 at 125M to 0.767 at 6.7B for full fine-tuning; more epochs and more data (25% to 100%) shift attack AUC in the expected directions.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use full fine-tuning instead of LoRA | LLM adaptation on CorpClimate · CorpClimate | AUC-ROC: 0.735 -> 0.882 | Helped |
| Use full fine-tuning instead of LoRA | LLM adaptation on CorpClimate · CorpClimate | accuracy: 0.674 -> 0.677 | No clear effect |
| Use in-context learning without parameter updates instead of full fine-tuning | Llama-7B · CorpClimate | AUC-ROC: 0.882 -> 0.922 | Helped |
| Use in-context learning without parameter updates instead of full fine-tuning | Llama-7B · CorpClimate | accuracy: 0.677 -> 0.534 | Hurt |
| Adapt with more data, from 25%(2790) to full(11159) (ablation) | Full fine-tuning on OPT-6.7B · CorpClimate | Attack AUC: 0.794 -> 0.751 | Hurt |
| Adapt with more data, from 25%(2790) to full(11159) (ablation) | Full fine-tuning on OPT-6.7B · CorpClimate | Model Accuracy: 0.424 -> 0.674 | Helped |
| Increase fine-tuning from 1 to 5 epochs (ablation) | Full fine-tuning on OPT-6.7B · CorpClimate | Attack AUC: 0.679 -> 0.755 | Helped |
| Increase fine-tuning from 1 to 5 epochs (ablation) | Full fine-tuning on OPT-6.7B · CorpClimate | Model Accuracy: 0.404 -> 0.673 | Helped |
| Increase model size from OPT-125M to OPT-6.7B (ablation) | Full fine-tuning on CorpClimate · CorpClimate | Attack AUC-ROC: 0.699 -> 0.767 | Helped |
| Increase model size from OPT-125M to OPT-6.7B (ablation) | Full fine-tuning on CorpClimate · CorpClimate | Model Accuracy: 0.669 -> 0.685 | Helped |
| Use Likelihood-ref rather than other evaluated attacks | Membership inference on adapted LLMs · Sujet Finance | AUC-ROC: 0.73 | Mixed |
Surprising findings
- In-context learning, which performs no parameter updates, still yields non-trivial attack AUC (e.g., 0.881-0.922 on CorpClimate for T5-Large and Llama-7B in Table 1, though the table is labeled with the same values across attacks).
- Structured SQL data is harder to attack; misclassified non-members look structurally identical to members.
Limitations
- Evaluates only membership inference with black-box and white-box attacks; other attacks such as data extraction or attribute inference are not covered.
- Does not evaluate differentially private adaptation; results depend on chosen datasets, architectures, and attack settings.
- Attack AUCs are modest (mostly 0.5-0.75), so results are a lower bound on worst-case risk.
Experimental setup
Datasets: Sujet-Finance-Instruct-177k Corporate Climate Policy Engagement (CorpClimate) Synthetic-Text-to-SQL
Benchmarks: Sujet Finance CorpClimate Synthetic-Text-to-SQL
Metrics: AUC-ROC FPR@0.1%TPR FPR@1%TPR model accuracy
Baselines: Likelihood Likelihood-ref Zlib Entropy Neighborhood Min-K% Min-K%++ Gradient Norm
Builds on: LoRA Min-K% Probability Likelihood-ratio membership inference (Carlini et al.) Neighborhood attack Prefix-tuning P-tuning Prompt-tuning Adapter
Closest papers by meaning
- Open LLMs are Necessary for Current Private Adaptations and Outperform their Closed Alternatives (NeurIPS 2024 · similarity 0.93)
- Privacy Auditing of Large Language Models (ICLR 2025 · similarity 0.91)
- Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory (ICLR 2024 · similarity 0.90)
- Forget to Flourish: Leveraging Machine-Unlearning on Pretrained Language Models for Privacy Leakage (AAAI 2025 · similarity 0.89)
- The Canary’s Echo: Auditing Privacy Risks of LLM-Generated Synthetic Text (ICML 2025 · similarity 0.89)
- Scaling Laws for Differentially Private Language Models (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.