Research map / Large language models / Instruction and fine-tuning
Efficient and Privacy-Preserving Soft Prompt Transfer for LLMs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes POST, a framework that distills a small proxy from an LLM, tunes soft prompts privately (optionally with DP) on the proxy, and transfers them to the large LLM using only a small public dataset.
Problem
Soft prompts are tightly coupled to the LLM they were tuned on, so tuning them on large hosted LLMs is computationally expensive and requires sharing private data with the provider; existing soft prompt transfer either leaks private data to the provider or underperforms the small prompted model.
Approach
Distill a small model from the large LLM via knowledge distillation so its feature space aligns with the teacher; tune the private soft prompt locally on the small model (with PromptDPSGD optionally); then transfer it to the large LLM by matching output and output-change distributions on a small public dataset, with no private data access.
Key results
- On Llama2-7b without DP, POST transfer reaches 90.02 on sst2 (imdb public) vs 78.78 for Compressed PT and 55.28 for Direct Transfer.
- With DP (epsilon=8), POST on Llama2-7b reaches 89.91 on sst2 and 80.17 on mpqa, outperforming DP-OPT and zero-shot transfer baselines.
- On sst2 with Llama2-7b, POST reduces tuning runtime from 2,660 to 409 minutes on an A100 (excluding KD), a sixfold speedup.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| transfer the prompt tuned on a small model to Llama2-7b using public data | POST · sst2 | accuracy: 55.28 -> 90.02 | Helped |
| transfer the prompt tuned on a small model to Llama2-7b using public data | POST · tweet | accuracy: 41.70 -> 79.59 | Helped |
| transfer the prompt tuned on a small model to Llama2-7b using public data | POST · arisetv | accuracy: 54.23 -> 86.71 | Helped |
| transfer the prompt tuned on a small model to Llama2-7b using public data | POST · mpqa | accuracy: 32.96 -> 87.37 | Helped |
| apply differential privacy during source prompt tuning | POST · sst2 | accuracy: 90.02 -> 89.91 | Hurt |
| apply differential privacy during source prompt tuning | Llama2-7b soft prompts · sst2 membership inference | AUC: 0.5610 -> 0.5258 | Helped |
| replace knowledge distillation with direct fine-tuning on BookCorpus (ablation) | POST transfer using a 2-layer Llama model · sst2 | accuracy: 90.02 -> 85.38 | Hurt |
| replace knowledge distillation with direct fine-tuning on BookCorpus (ablation) | POST transfer using a 2-layer Llama model · imdb | accuracy: 87.29 -> 82.45 | Hurt |
| replace knowledge distillation with direct fine-tuning on BookCorpus (ablation) | POST transfer using a 2-layer Llama model · tweet | accuracy: 61.15 -> 45.65 | Hurt |
| use the three-term distillation loss instead of a single-term or two-term loss (ablation) | POST with Roberta-base · sst2 | accuracy: 85.32, 78.21, 76.38, 85.67, 84.75, 85.67 -> 87.73 | Helped |
| use the three-term distillation loss instead of a single-term or two-term loss (ablation) | POST with Roberta-base · tweet | accuracy: 53.65, 47.05, 45.80, 54.95, 55.05, 50.40 -> 58.25 | Helped |
| use a same-task-family public dataset rather than a different-family dataset (ablation) | POST · arisetv | accuracy: 57.13 -> 87.56 | Helped |
Surprising findings
- Transferred prompts with DP sometimes improve over the non-DP transfer gap relative to the compressed model, which the authors hypothesize is regularization from DP noise.
- Public data from the same task family works best even when class counts differ (e.g., imdb with tweet as public set).
Limitations
- Does not address privacy leakage during inference (private inference is out of scope).
- Does not address pretraining data leakage from distillation; authors say a more thorough investigation is warranted.
- No principled method for selecting the public dataset; transfer performance depends on it.
Experimental setup
Datasets: BookCorpus sst2 imdb tweeteval arisetv mpqa MIT movie agnews boolq AIE disaster trec
Benchmarks: sst2 imdb tweeteval arisetv mpqa MIT-D MIT-G disaster trec
Metrics: test accuracy AUC TPR@1%FPR runtime (minutes)
Baselines: Full ZS Full PT Compressed PT Direct Transfer OPT DP-OPT Zero-Shot Transfer
Builds on: PromptDPSGD knowledge distillation (DistilBERT-style losses) prompt tuning DP-OPT zero-shot prompt transfer
Closest papers by meaning
- DP-OPT: Make Large Language Model Your Privacy-Preserving Prompt Engineer (ICLR 2024 · similarity 0.92)
- Soft Prompt Recovers Compressed LLMs, Transferably (ICML 2024 · similarity 0.89)
- Privacy-Preserving Instructions for Aligning Large Language Models (ICML 2024 · similarity 0.88)
- Open LLMs are Necessary for Current Private Adaptations and Outperform their Closed Alternatives (NeurIPS 2024 · similarity 0.87)
- Hidden No More: Attacking and Defending Private Third-Party LLM Inference (ICML 2025 · similarity 0.87)
- Breaking Physical and Linguistic Borders: Multilingual Federated Prompt Tuning for Low-Resource Languages (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.