Research map / Large language models / Instruction and fine-tuning
Synthetic Text Generation for Training Large Language Models via Gradient Matching
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GrADmm, an ADMM-based method that optimizes synthetic text embeddings to match (DP-noised) last-layer gradients of real data, projects them to readable low-perplexity tokens, and filters them, with convergence guarantees for fine-tuning LLMs.
Problem
Generating small, human-readable, privacy-preserving synthetic text that makes fine-tuning an LLM mimic training on real data, with performance guarantees; existing text synthesis is heuristic, LLM-prompting based, or yields unreadable embeddings.
Approach
Formulate synthetic text generation as discretely constrained gradient matching, relax it into embedding space and solve with ADMM (Adam for the primal step, top-k vocabulary projection for the constraint), match only last-layer gradients, add Gaussian noise to clipped real-data gradients for DP, and filter outputs; theory shows fine-tuning on the synthetic set stays close to fine-tuning on real d
Key results
- Synthetic data from only 5 real SST-2 validation examples outperforms training on the real examples by 15.7%, 29.7%, and 31.5% on SST-2, Tweet emotions, and Rotten tomatoes.
- GrADmm outperforms zero-shot, few-shot, and coreset baselines across datasets and sizes by up to 13.1%, reaching 89.7% on SST-2 with 50-100 examples (non-DP).
- Synthetic data from Phi transfers to Llama-3.2-1B and OPT-1.3B, e.g. Llama-3.2-1B on SST-2 reaches 89.4 vs 82.4 zero-shot.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Generate synthetic data with GrADmm rather than zero-shot generation | Phi fine-tuning · SST-2, 20 examples | accuracy: 82.2 -> 87.9 | Helped |
| Generate synthetic data with GrADmm rather than zero-shot generation | Phi fine-tuning · Tweet emotions, 20 examples | accuracy: 81.7 -> 85.2 | Helped |
| Generate synthetic data with GrADmm rather than zero-shot generation | Phi fine-tuning · Rotten tomatoes, 20 examples | accuracy: 78.0 -> 84.4 | Helped |
| Transfer Phi-generated synthetic data instead of random real samples | Llama-3.2-1B · SST-2, 20 samples | accuracy: 88.4 -> 89.4 | Helped |
| Transfer Phi-generated synthetic data rather than K-center samples | Llama-3.2-1B · Tweet emotions, 20 samples | accuracy: 88.5 -> 85.8 | Hurt |
| Transfer Phi-generated synthetic data rather than K-center samples | Llama-3.2-1B · Rotten tomatoes, 20 samples | accuracy: 87.8 -> 87.8 | No clear effect |
| Match full gradients instead of last-layer gradients (ablation) | GrADmm · SST-2 | accuracy: 90.0 -> 89.6 | Hurt |
| Remove top-k projection (ablation) | GrADmm · SST-2 | accuracy: 90.0 -> 80.8 | Hurt |
| Remove examples with incorrect labels (ablation) | ADMM synthetic data · SST-2 | accuracy: 88.1 -> 89.4 | Helped |
| Select the lowest-loss synthetic samples (ablation) | Filtered ADMM data · SST-2 | accuracy: 89.4 -> 89.4 | No clear effect |
| Balance category losses by dropping the highest-loss samples (ablation) | Filtered ADMM data · SST-2 | accuracy: 89.4 -> 90.0 | Helped |
| Increase ε to relax the privacy constraint (ablation) | GrADmm synthetic data · Classification tasks | accuracy | Helped |
Surprising findings
- With 5 validation examples, the DP version (epsilon=4) outperforms the non-DP version, which the authors hypothesize is noise-induced generalization.
- Top-k projection yields readable text but the non-DP and DP versions show MIA advantage near zero, suggesting privacy is preserved even without DP.
Limitations
- Theory relies on smoothness and PL* assumptions and a linearized/near-pretrained approximation; bounds are loose.
- Experiments limited to short text classification tasks and a small set of models.
- Top-k projection reduces gradient matching fidelity, requiring filtering; synthetic text is not always semantically clean.
Experimental setup
Datasets: SST-2 Tweet emotions Rotten tomatoes IMDB Sentence polarity
Benchmarks: SST-2 Tweet emotions Rotten tomatoes IMDB Sentence polarity
Metrics: classification accuracy log-perplexity FID gradient error membership inference advantage
Baselines: zero-shot LLM generation few-shot LLM generation Herding K-center Random coreset
Builds on: dataset distillation ADMM top-k decoding DP-SGD
Closest papers by meaning
- PrE-Text: Training Language Models on Private Federated Data in the Age of LLMs (ICML 2024 · similarity 0.88)
- Idiosyncrasies in Large Language Models (ICML 2025 · similarity 0.87)
- Not All LLM-Generated Data Are Equal: Rethinking Data Weighting in Text Classification (ICLR 2025 · similarity 0.87)
- Escaping Collapse: The Strength of Weak Data for Large Language Model Training (NeurIPS 2025 · similarity 0.86)
- Controlled Text Generation via Language Model Arithmetic (ICLR 2024 · similarity 0.86)
- Towards a Theoretical Understanding of Synthetic Data in LLM Post-Training: A Reverse-Bottleneck Perspective (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.