Research map / Large language models / Language, safety and interpretability

How to Synthesize Text Data without Model Collapse?

ICML 2025 · Large language models › Language, safety and interpretability · approach: Controlled text generation · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

The paper shows that mixing synthetic data into pre-training degrades performance (non-iterative model collapse) and proposes token-level editing (ToEdit), which resamples high-confidence tokens of human data using a prior LM to create semi-synthetic data, with a theoretical bound on test error.

Problem

Training language models on a mixture of synthetic and human-produced data degrades performance (non-iterative model collapse), and it is unclear how to synthesize text data without causing model collapse.

Approach

Instead of generating fully synthetic text, token-level editing resamples tokens in human-written data whose conditional probability under a prior LM exceeds a threshold, preserving the human distribution's long tail while improving data; a linear-model analysis shows the test error stays bounded across editing iterations.

Key results

• Pre-training GPT-2 Small on increasing synthetic proportions raises average PPL over 22 subdomains from 20.99 (human) to 51.93 (pure synthetic).
• ToEdit improves OLMo-1B pre-training average across 8 general tasks from 32.75 to 33.11.
• ToEdit improves Llama-3-8B biomedicine CPT average from 56.04 to 56.48 and OLMo-1B biomedicine average from 38.83 to 40.89; FLAN v2 SFT average on Llama-3-8B rises from 70.18 to 70.65.

Measured effects reported in the paper

Surprising findings

• Lower training loss on synthetic data does not translate into better downstream performance, as models fitting synthetic data better perform worse on real tasks.
• Even DSIR importance-sampling data selection fails to correct the distribution shift of synthetic data.

Limitations

• Improvements are modest (often under 1 point on general tasks), and some tasks show drops.
• Theoretical analysis is restricted to a linear ridgeless regression model with isotropic features and a geometric assumption on editing rates.

Experimental setup

Datasets: Dolma Cosmopedia Paloma The Pile FLAN v2 CoT Open Assistant 1 OSS-Instruct-75K Evol-Instruct-110K PubMed Abstracts financial news (FinGPT)

Benchmarks: Paloma PIQA BoolQ OpenBookQA ARC-c ARC-e HellaSwag SIQA Winogrande PubMedQA ChemProt MQP

Metrics: perplexity accuracy

Baselines: Cosmopedia synthetic data Llama-3 natural instructions OLMo-1B pre-trained CPT on original data top-p sampling rejection sampling DSIR data selection

Builds on: Dolma Cosmopedia DSIR AdaptLLM domain-adaptive continual pre-training

Closest papers by meaning

• Towards a Theoretical Understanding of Synthetic Data in LLM Post-Training: A Reverse-Bottleneck Perspective (ICLR 2025 · similarity 0.89)
• Collapse or Thrive: Perils and Promises of Synthetic Data in a Self-Generating World (ICML 2025 · similarity 0.89)
• Beyond Model Collapse: Scaling Up with Synthesized Data Requires Verification (ICLR 2025 · similarity 0.88)
• It's All Training: A Fully Synthetic Single-Stage Recipe for LLMs (NeurIPS 2026 · similarity 0.88)
• Escaping Collapse: The Strength of Weak Data for Large Language Model Training (NeurIPS 2025 · similarity 0.88)
• Maximizing the Potential of Synthetic Data: Insights from Random Matrix Theory (ICLR 2025 · similarity 0.86)