Research map / Federated learning and privacy / Differential privacy
Differentially Private Synthetic Data via Foundation Model APIs 2: Text
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Aug-PE, an API-only differentially private synthetic text method that runs the Private Evolution loop with LLM variation (rephrase, fill-in-blank) and DP nearest-neighbor histograms over embeddings, without fine-tuning.
Problem
Generating differentially private synthetic text is hard when LLMs are closed-source (no weight access) and DP-SGD fine-tuning of open LLMs is computationally expensive due to per-sample clipping and noise.
Approach
Adapts the Private Evolution framework to text: initial samples from an API conditioned on labels/keywords, then iterative LLM-based variations (paraphrase, fill-in-blanks) selected by a DP nearest-neighbor histogram over embeddings of private data, so only API calls and noisy histograms touch private data.
Key results
- On Yelp with epsilon=4, GPT-3.5 fill-in-blank with 5k synthetic samples reaches 0.6781 rating accuracy, close to the DP fine-tuned GPT-2 (100k samples) at 0.6656.
- On Yelp, API fill-in-blank with GPT-3.5 gives FID 16.32 at epsilon=4 versus 11.47 for DP fine-tuning, indicating a distributional gap.
- On ICLR 2023 reviews, API fill-in-blank (GPT-3.5, 2k samples) reaches 0.4521 area accuracy at epsilon=infinity, well below the original data's 0.6515.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use API new-vote selection instead of standard API rephrase (ablation) | GPT2 API generation · Yelp, ε=∞ | Rating accuracy: 0.542 -> 0.6645 | Helped |
| Use API new-vote selection instead of standard API rephrase (ablation) | GPT2 API generation · Yelp, ε=∞ | FID: 20.65 -> 18.06 | Helped |
| Use API new-vote selection instead of standard API fill in blanks (ablation) | GPT3.5 API generation · Yelp, ε=∞ | Rating accuracy: 0.6803 -> 0.6837 | Helped |
| Use API new-vote selection instead of standard API fill in blanks (ablation) | GPT3.5 API generation · Yelp, ε=∞ | Category accuracy: 0.7415 -> 0.7285 | Hurt |
| Use API new-vote selection instead of standard API fill in blanks (ablation) | GPT3.5 API generation · Yelp, ε=∞ | FID: 16.32 -> 25.90 | Hurt |
| Increase generation temperature (ablation) | GPT3.5 API fill-in-mask generation · Yelp | FID | Helped |
| Increase generation temperature (ablation) | GPT3.5 API fill-in-mask generation · Yelp | generation diversity | Helped |
| Increase mask probability (ablation) | GPT3.5 API fill-in-mask generation · ICLR 2023 Reviews | generation diversity | Helped |
| Increase lookahead k (ablation) | GPT2 API rephrase generation · Yelp | FID | Helped |
Surprising findings
- The API-based method with only 5k synthetic samples approaches DP fine-tuned GPT-2 trained on 100k samples in downstream accuracy on Yelp at epsilon=4.
- GPT-3.5 outperforms GPT-2 for the API-based generation in the reported experiments.
Limitations
- Synthetic text distribution is still further from private data than DP fine-tuning on FID metrics.
- Some experiments are incomplete (missing table entries and a TODO to enlarge the dataset), and the draft lacks a full privacy analysis.
- Requires sequences of sufficient length; short outputs caused API call failures.
Experimental setup
Datasets: Yelp review data ICLR 2023 reviews
Benchmarks: Yelp review rating classification Yelp business category classification ICLR 2023 review area and rating classification
Metrics: accuracy FID F1 MAUVE
Baselines: DP-SGD fine-tuning of GPT-2
Builds on: Private Evolution DP-SGD
Closest papers by meaning
- Differentially Private Synthetic Data via Foundation Model APIs 1: Images (ICLR 2024 · similarity 0.93)
- SynBench: A Benchmark for Differentially Private Text Generation (NeurIPS 2026 · similarity 0.91)
- PrE-Text: Training Language Models on Private Federated Data in the Age of LLMs (ICML 2024 · similarity 0.89)
- PCEvolve: Private Contrastive Evolution for Synthetic Dataset Generation via Few-Shot Private Data and Generative APIs (ICML 2025 · similarity 0.89)
- Private Evolution Converges (NeurIPS 2025 · similarity 0.89)
- Synthesize Privacy-Preserving High-Resolution Images via Private Textual Intermediaries (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.