Research map / Large language models / Language, safety and interpretability
Beyond Model Collapse: Scaling Up with Synthesized Data Requires Verification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that verifier-based selection of LLM-synthesized data prevents model collapse, with a theory of Gaussian-mixture pruning, a breakdown-point proxy p*, and experiments on eigenvalue transformers and Llama-2 news summarization.
Problem
Training on model-generated data causes model collapse (performance drops below the generator), breaking neural scaling laws; the question is whether synthesized data can be salvaged by selection.
Approach
Treat synthesized data as labels and apply verifier-based pruning; characterize with a phase transition in the infinite-sample limit that good accuracy is achievable as long as the label corruption rate is below a breakdown point determined by verifier quality, and use this breakdown point p* as a measurable proxy for final performance.
Key results
- On the eigenvalue task, the oracle verifier nearly doubles accuracy, while random selection without verification (p*=0.5) stays below the generator even with 10M samples.
- On news summarization, oracle selection with 12.5% of synthesized data surpasses the model trained on 100% of original labels.
- Self-selection by perplexity outperforms the generator, while Llama-3 weak-supervision selection performs similarly to random selection despite higher Rouge-1, as predicted by the proxy p*.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the most accurate candidate among 50 beams rather than the single greedy candidate | Transformer eigenvalue generator · Eigenvalue prediction, τ=1% | accuracy: 20.2 -> 60.4 | Helped |
| Increase beam size while evaluating only the model-ranked best candidate | Transformer eigenvalue generator · Eigenvalue prediction, τ=1% (best beam) | accuracy: 20.2 -> 19.2 | No clear effect |
| Fine-tune the generator on its own generated data (ablation) | Eigenvalue prediction transformer · Eigenvalue prediction, τ=2%, 1M examples | accuracy: 66% -> 61.8 | Hurt |
| Fine-tune a model pretrained to higher accuracy than the generator using generated data (ablation) | Eigenvalue prediction transformer · Eigenvalue prediction, τ=2%, 1M examples | accuracy: 93% -> 82.1 (rejection); 93% -> 87.3 (Beam 50) | Helped |
| Fine-tune models pretrained at the generator's accuracy or lower using generated data (ablation) | Eigenvalue prediction transformer · Eigenvalue prediction, τ=2%, 1M examples | accuracy | No clear effect |
| Select synthesized arithmetic examples with an oracle verifier | Transformer trained from scratch · Eigenvalue prediction | accuracy | Helped |
| Train on unverified synthesized arithmetic data instead of verified data | Transformer trained from scratch · Eigenvalue prediction, 10 million samples | accuracy | Hurt |
| Use oracle verification instead of unverified random selection | Transformer trained from scratch · Eigenvalue prediction | accuracy | Helped |
| Select synthesized summaries by oracle Rouge-1 scores | Llama-2-7B · XLSUM | Rouge-1 | Helped |
| Select Llama-2-generated summaries using Llama-3 verification | Llama-2-7B · XLSUM | Rouge-1 | No clear effect |
| Self-select Llama-2-generated summaries by lowest perplexity | Llama-2-7B · XLSUM | Rouge-1 | Helped |
| Increase synthetic dataset size using random selection | Llama-2-7B · XLSUM | Rouge-1 | Mixed |
Surprising findings
- A stronger verifier (Llama-3, Rouge-1 34.5) gives performance similar to random selection, while self-selection by perplexity works better; the verifier's alignment with the generator matters more than its raw quality.
- Beam search with oracle selection over 50 beams reaches 60.4% accuracy versus 20.2% for greedy, yet perplexity-based selection of the best beam does not improve accuracy.
Limitations
- Only data selection studied; augmentation, regeneration and weighting not explored.
- Theory limited to infinite-sample, high-dimensional Gaussian-mixture setting with linear classifiers and verifiers.
- Synthesized data are used as labels with fixed inputs in the main theory; the Llama experiment also shifts input distribution, complicating attribution.
Experimental setup
Datasets: XLSUM English summarization synthetic random 5x5 symmetric matrices
Benchmarks: XLSUM English test set eigenvalue test set with relative L1 tolerance
Metrics: accuracy with tolerance Rouge-1 classification accuracy (simulation) breakdown point p*
Baselines: random selection without verification generator (Llama-2-7B) Llama-3 weak supervision selection self-selection by perplexity
Builds on: model collapse beam search XLSUM
Closest papers by meaning
- Escaping Collapse: The Strength of Weak Data for Large Language Model Training (NeurIPS 2025 · similarity 0.90)
- Strong Model Collapse (ICLR 2025 · similarity 0.90)
- An Empirical Study on Noisy Data and LLM Pretraining Loss Divergence (NeurIPS 2026 · similarity 0.89)
- How to Synthesize Text Data without Model Collapse? (ICML 2025 · similarity 0.88)
- LLMs on the Line: Data Determines Loss-to-Loss Scaling Laws (ICML 2025 · similarity 0.88)
- Measuring what Matters: Construct Validity in Large Language Model Benchmarks (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.