Research map / Federated learning and privacy / Privacy attacks and secure learning
Synthesizing Privacy-Preserving Text Data via Finetuning *without* Finetuning Billion-Scale LLMs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CTCL, which pretrains a 140M BART-based conditional generator and a Wikipedia-based topic model, then DP-finetunes the generator on private data and samples it using a DP topic histogram to synthesize privacy-preserving text without billion-scale LLM finetuning or prompt engineering.
Problem
Generating privacy-preserving synthetic text is either computationally expensive (DP finetuning of billion-scale LLMs) or depends on manual prompts and under-uses private data (prompt-based methods like Private Evolution), which hurts generative tasks.
Approach
Combine a lightweight 140M keyword-conditioned generator, continually pretrained on 430M LLM-described SlimPajama (condition, document) pairs, with a Wikipedia-clustered topic model; DP-finetune the generator on keyword-conditioned private data and sample per topic according to a DP topic histogram, so synthetic size is unconstrained at no extra privacy cost.
Key results
- On PubMed at epsilon=1, CTCL reaches 34.5% next-word accuracy with BERT-mini versus 30.1% for AUG-PE+GPT-3.5 and 29.8% for PE.
- On OpenReview at epsilon=1, CTCL achieves 46.2% classification accuracy versus 43.1% for AUG-PE and 25.7% for BART-base DP finetuning.
- CTCL scales with privacy budget and synthetic size, while AUG-PE does not benefit from larger epsilon and plateaus beyond 10K examples.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CTCL instead of the strongest listed baseline | PubMed next-word prediction with BART-base downstream model · PubMed, ε=4 | next-word prediction accuracy: 35.3 -> 35.9 | Helped |
| Use CTCL instead of the strongest listed baseline | PubMed next-word prediction with BERT-small downstream model · PubMed, ε=1 | next-word prediction accuracy: 32.4 -> 36.7 | Helped |
| Use CTCL instead of the strongest listed baseline | Yelp classification · Yelp, ε=∞ | accuracy: 76.0 -> 70.5 | Hurt |
| Use CTCL instead of the strongest listed baseline | Yelp classification · Yelp, ε=4 | accuracy: 68.2 -> 68.1 | No clear effect |
| Use CTCL instead of the strongest listed baseline | OpenReview classification · OpenReview, ε=2 | accuracy: 43.7 -> 47.1 | Helped |
| Add keyword-based conditions during DP finetuning (ablation) | BART-base generator for PubMed synthesis · PubMed, ε=∞ | perplexity: 63.1 -> 49.3 | Helped |
| Add keyword-based conditions during DP finetuning (ablation) | BART-base generator for PubMed synthesis · PubMed, ε=1 | perplexity: 572.9 -> 291.6 | Helped |
| Add generator pretraining to keyword-conditioned finetuning (ablation) | BART-base generator for PubMed synthesis · PubMed, ε=∞ | perplexity: 49.3 -> 48.4 | Helped |
| Add generator pretraining to keyword-conditioned finetuning (ablation) | BART-base generator for PubMed synthesis · PubMed, ε=1 | perplexity: 291.6 -> 125.6 | Helped |
Surprising findings
- Higher MAUVE similarity did not always translate to better downstream performance: GPT-2 XL synthetic data had lower MAUVE than CTCL but gave higher downstream accuracy (37.7% vs 35.9% on PubMed).
- Fluent GPT-3.5 synthetic text from AUG-PE performed only comparably to DP-finetuned BART-base, suggesting surface fluency is not the most critical factor for downstream utility.
Limitations
- Synthetic text from the 140M generator lacks the fluency and coherence of billion-scale LLM outputs.
- Pretraining data curation requires running a 2B LLM over 430M documents, which is costly once.
- Results on the strong-privacy regime rely on a topic-histogram privacy allocation; the approach is text-only and not yet extended to multimodal data.
Experimental setup
Datasets: SlimPajama Wikipedia (Nov 2023) RedPajama PubMed Chatbot Arena Multi-Session Chat Yelp OpenReview
Benchmarks: PubMed Chatbot Arena Multi-Session Chat Yelp OpenReview
Metrics: next-word prediction accuracy classification accuracy perplexity MAUVE
Baselines: Downstream DPFT GPT-2 XL DP finetuning GPT-2 XL LoRA DP finetuning GPT-2 small DP finetuning BART-base DP finetuning Private Evolution AUG-PE Post-Generation Resampling
Builds on: Private Evolution BART BERTopic Post-Generation Resampling DP-Adam
Closest papers by meaning
- SynBench: A Benchmark for Differentially Private Text Generation (NeurIPS 2026 · similarity 0.91)
- PrE-Text: Training Language Models on Private Federated Data in the Age of LLMs (ICML 2024 · similarity 0.89)
- Privacy-Preserving Instructions for Aligning Large Language Models (ICML 2024 · similarity 0.89)
- InvisibleInk: High-Utility and Low-Cost Text Generation with Differential Privacy (NeurIPS 2025 · similarity 0.88)
- Struct-Bench: A Benchmark for Differentially Private Structured Text Generation (NeurIPS 2025 · similarity 0.88)
- Open LLMs are Necessary for Current Private Adaptations and Outperform their Closed Alternatives (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.