Research map / Federated learning and privacy / Privacy attacks and secure learning
Open LLMs are Necessary for Current Private Adaptations and Outperform their Closed Alternatives
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Compares four recent private in-context-learning methods for closed LLMs against private gradient-based adaptation (PrivateLoRA, DP-FineTune, PromptDPSGD) on open LLMs, analyzing privacy leakage, performance, and cost, and proposes private prompt tuning/PATE variants for text generation.
Problem
Whether privately adapting closed LLMs via discrete-prompt private ICL protects privacy (training and query data) and performs well compared to private adaptation of open local LLMs.
Approach
Argues that private gradient-based tuning of open LLMs is more private (query data never leaves the data owner), more performant, and cheaper than private ICL on closed LLMs; extends PromptDPSGD and PromptPATE to text generation (PromptDPSGDGen, PromptPATEGen).
Key results
- On SST2 with epsilon=8, PrivateLoRA on Llama3-8B reaches 96.0% accuracy at about $28 total cost versus DP-ICL on GPT4 Turbo at 95.9% with $138 query cost for 10k queries.
- On average over four classification tasks, PrivateLoRA on Vicuna 7B reaches 90.3 vs 82.9 for non-private ICL on GPT3 Davinci and 81.4 for DP-OPT on Davinci (epsilon=8).
- On SAMSum, PrivateLoRA on BART-Large achieves RougeL 39.1 vs 31.8 for DP-ICL on GPT3 Davinci, with total cost $3.59 vs $665.91 for 10k queries.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PrivateLoRA on Vicuna 7B instead of PromptPATE on GPT3 Babbage at ε=0.1 | Private classification adaptation · SST2, Trec, Disaster | accuracy: 0.6%; 4.4%; 3.5% | Helped |
| Use PromptPATE instead of PrivateLoRA | Private classification adaptation · Mpqa | accuracy | Mixed |
| Use open local gradient-based adaptation instead of closed-model private ICL | Private classification adaptation · Trec | accuracy: even >26% absolute | Helped |
| Increase the privacy budget for PromptPATE beyond around ε=0.3 | PromptPATE · Classification tasks | accuracy | No clear effect |
| Use PrivateLoRA on BART-Large instead of DP-ICL on GPT3 Davinci | Private summarization adaptation · SAMSum | Rouge-L: 31.8 -> 39.1 | Helped |
| Use PromptDPSGDGen instead of PrivateLoRA | Private generation adaptation · SAMSum | Rouge-L: 39.1 -> 37.4 | Hurt |
| Use open local gradient-based adaptation instead of DP-ICL | Private generation adaptation · PFL-DocVQA | Rouge-1: 60.7 -> 70.2 | Helped |
| Use PrivateLoRA on OpenLLaMA 13B instead of FewShotGen on GPT3 Davinci | Private information extraction · MIT-D, MIT-G | accuracy: 13.4%; 22.3% absolute | Helped |
| Increase soft-prompt parameters from 0.1% to 10% using prefix projection | PromptDPSGDGen · Text generation | performance | Helped |
| Use a higher learning rate for prompt tuning | PromptDPSGDGen · Text generation | performance | Helped |
| Use open local LLM adaptations instead of closed-model private adaptations | Private LLM adaptation · SAMSum | query cost: 665.91 -> 3.59 | Helped |
Surprising findings
- Even non-private ICL with GPT3 Davinci underperforms DP-FineTune and PrivateLoRA on RoBERTa-Large (355M) on classification tasks.
- PromptPATE on closed models plateaus after around epsilon=0.3, and private LoRA on a 355M-parameter model beats DP-ICL on a 175B closed model on dialog summarization.
Limitations
- Hyperparameters were not tuned for all open-model private tuning methods due to compute limits.
- Closed-model experiments limited to four closed models due to budget; GPT4 Turbo not extended to all tasks.
- Some larger models (13B for PrivateLoRA classification, 7B+ for PromptDPSGDGen and DP-FineTune generation) could not be trained due to library incompatibilities.
Experimental setup
Datasets: SST2 Trec Mpqa Disaster SAMSum PFL-DocVQA MIT Movies (MIT-D, MIT-G)
Benchmarks: SST2 Trec Mpqa Disaster SAMSum PFL-DocVQA MIT-D MIT-G
Metrics: accuracy ROUGE-1 ROUGE-2 ROUGE-L BLEU Levenshtein monetary cost (training and query)
Baselines: DP-ICL PromptPATE DP-FewShotGen DP-OPT DP-FineTune PromptDPSGD PrivateLoRA
Builds on: PromptDPSGD PrivateLoRA DP-ICL PromptPATE DP-FewShotGen DP-OPT PATE
Closest papers by meaning
- PrivAuditor: Benchmarking Data Protection Vulnerabilities in LLM Adaptation Techniques (NeurIPS 2024 · similarity 0.93)
- Privacy Auditing of Large Language Models (ICLR 2025 · similarity 0.90)
- Scaling Laws for Differentially Private Language Models (ICML 2025 · similarity 0.90)
- Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory (ICLR 2024 · similarity 0.90)
- Differentially Private Federated Low Rank Adaptation Beyond Fixed-Matrix (NeurIPS 2025 · similarity 0.89)
- Hidden No More: Attacking and Defending Private Third-Party LLM Inference (ICML 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.