Research map / Large language models / Language, safety and interpretability

Data Mixing Can Induce Phase Transitions in Knowledge Acquisition

NeurIPS 2025 · Large language models › Language, safety and interpretability · approach: LLM pruning and scaling data · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that mixing a small knowledge-dense synthetic biography dataset into web text induces sharp phase transitions in knowledge memorization with respect to model size and mixing ratio, explains them with a capacity-allocation (knapsack-like) information-theoretic model, and proposes subsampling and compact knowledge mixing to improve acquisition.

Problem

Knowledge acquisition from small knowledge-dense datasets mixed into web-scraped pre-training data does not follow the linear scaling law seen when training on knowledge-dense data alone; the paper asks how memorization depends on model size and mixing ratio.

Approach

Mixing knowledge-dense data with web data makes the optimal bounded-capacity learner allocate capacity by marginal loss reduction (a fractional knapsack view), so the learned amount jumps discontinuously in model size and mixing ratio; the threshold scales as a power law in model size, and this is exploited via subsampling and compact rephrasing.

Key results

• Pythia 410M models at mixing ratio r<=0.3 stay near zero accuracy on SynBio, while r=0.4 reaches roughly 80% accuracy.
• Required training steps to reach 40% accuracy grow faster than the power-law fit predicts for r=0.25 (more than 2.9x the power-law prediction).
• Subsampling SynBio-1.28M to 50% at r=0.2 raises 410M SynBio accuracy from near zero to 37.46% while keeping downstream performance within 2%.

Measured effects reported in the paper

Surprising findings

• Randomly subsampling the knowledge-dense data can increase the number of memorized facts, since each fact gets higher exposure frequency.
• Naive small-model measurements on a small domain may give little signal about larger models' performance on the same domain.

Limitations

• Limited random-seed replication due to high compute cost; main results are single-seed.
• Mainly synthetic biographies as proxy for knowledge-dense data; real-world heterogeneous data may show less apparent transitions.
• Theory assumes a sufficiently trained optimal bounded-capacity learner and non-overlapping domains.

Experimental setup

Datasets: SynBio FineWeb-Edu The Pile OpenWebMath WikiBio PopQA

Benchmarks: SynBio PopQA LAMBADA ARC-E SciQ PIQA HellaSwag

Metrics: exact-match accuracy on memorized facts validation loss zero-shot downstream accuracy

Baselines: Pythia

Builds on: Pythia Allen-Zhu & Li knowledge capacity scaling (synthetic biographies) Chinchilla compute-optimal scaling

Closest papers by meaning

• Data Mixing Laws: Optimizing Data Mixtures by Predicting Language Modeling Performance (ICLR 2025 · similarity 0.89)
• Understanding LLM Behaviors via Compression: Data Generation, Knowledge Acquisition and Scaling Laws (NeurIPS 2025 · similarity 0.88)
• Physics of Language Models: Part 3.1, Knowledge Storage and Extraction (ICML 2024 · similarity 0.87)
• Knowledge Entropy Decay during Language Model Pretraining Hinders New Knowledge Acquisition (ICLR 2025 · similarity 0.87)
• How Do Large Language Models Acquire Factual Knowledge During Pretraining? (NeurIPS 2024 · similarity 0.87)
• Data Engineering for Scaling Language Models to 128K Context (ICML 2024 · similarity 0.87)