Research map / Large language models / Language, safety and interpretability
A Tale of Tails: Model Collapse as a Change of Scaling Laws
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Theoretical notes that model collapse from training on AI-generated data is equivalent to chopping off distribution tails, deriving scaling laws (Hutter-type, bigram, associative memory) for test error as a function of sample size, capacity, generation count, and clean-data mixing.
Problem
How model collapse from training on AI-generated (tail-truncated) data changes neural scaling laws, in toy Hutter-type, bigram, and associative-memory LLM settings.
Approach
Model AI-generated data as a finite-sample tail-chopped version of the true Zipf distribution (cutoff rank k tied to generator sample size T0), then derive scaling laws E_test ~ T^-c + k^-(beta-1) and show how multiple generations and mixing with clean data change them.
Key results
- Theory: with AI data cut at rank k, E_test scales as T^-(beta-1)/beta + k^-(beta-1), so additional AI samples stop helping once T exceeds k^beta.
- Mixing a fraction pi of clean data yields a grokking-like plateau of length ~k^beta/pi before error decays as (pi T)^-(1-1/beta).
- Multi-generation training gives E_n ~ T^-c (1 + n (T/T0)^c), so model collapse occurs when n is large relative to (T0/T)^c.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train on data truncated at rank k instead of the true distribution | Hutter model · Zipf-tailed data | test error: E_{test} \asymp T^{-(\beta-1)/\beta} + k^{-(\beta-1)} | Hurt |
| Use AI-generated data with tail exponent β′ rather than the true tail exponent β | Hutter model · Zipf-tailed data | test error: E_{test} \asymp T^{-(\beta-1)/\beta'} | Hurt |
| Limit conditional training data to the top k outcomes | Hutter++ · Autoregressive tailed bigram model | test error: E_{test} \asymp T^{-c} + k^{-\beta c} | Hurt |
| Add finite embedding capacity d | Associative memory · Zipf-tailed data | test error: E_{test} \asymp T^{-(1-1/\beta)} + d^{-c_q} + k^{-(\beta-1)} | Hurt |
| Use the seen-item indicator update instead of counting-measure updates | Associative memory · Capacity-limited model | capacity error exponent c_q: (1-1/\beta)/2 → 1-1/\beta | Helped |
| Mix a proportion π of clean data into tail-truncated training data | Hutter model · Clean and generated data mixture | test error: E_{test} \asymp (\pi T)^{-(1-1/\beta)} | Helped |
| Increase the number of AI-data generations n | Hutter model · Recursive AI data generation | test error: E^{(n)}_{test} \asymp T^{-c} + C_n T_0^{-c} | Hurt |
| Add an upsampled tail distribution with N/k → 1 | Hutter model · Tail-curated mixture | test error: E_{test} \asymp T^{-c} | Helped |
| Use a tail distribution with N/k → C > 1 instead of N/k → 1 | Hutter model · Tail-curated mixture | test error: E_{test} \asymp T^{-c} + k^{-\alpha} | Hurt |
| Apply top-p inference with p=0.9 when synthesizing data | Hutter++ · Multiple generations, β=3/2 | test error | Hurt |
| Apply temperature τ=0.9 when synthesizing data | Hutter++ · Multiple generations, β=3/2 | test error | Hurt |
Surprising findings
- Using an excessively deep tail (N much larger than k) in the mixture is worthless; only N close to k helps.
- Top-p 0.9 sampling significantly deteriorates collapse in some experiments, while temperature scaling creates strong collapse.
Limitations
- Results are for toy models (Hutter lookup, bigram, associative memory) with deterministic labels and Zipf distributions, not full transformers.
- Real-text experiments are only proposed, not reported in the notes.
- Many theorem statements and experiments are marked as drafts or TODO.
Experimental setup
Datasets: synthetic Zipf distribution samples synthetic permuted-Zipf bigram samples WikiText-103
Benchmarks: synthetic Zipf test error WikiText-103 perplexity
Metrics: test error total variation distance perplexity
Builds on: Hutter LLM (infinite memory model) Michaud et al. quantization model of skills Cui et al. associative memory scaling laws (CDB 2023)
Closest papers by meaning
- Strong Model Collapse (ICLR 2025 · similarity 0.93)
- Collapse or Thrive: Perils and Promises of Synthetic Data in a Self-Generating World (ICML 2025 · similarity 0.91)
- A Closer Look at Model Collapse: From a Generalization-to-Memorization Perspective (NeurIPS 2025 · similarity 0.89)
- A Hitchhiker's Guide to Scaling Law Estimation (ICML 2025 · similarity 0.89)
- On the Invariance and Generality of Neural Scaling Laws (NeurIPS 2026 · similarity 0.88)
- Scaling Laws for Pre-training Agents and World Models (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.