Research map / Large language models / Language, safety and interpretability
Understanding LLM Behaviors via Compression: Data Generation, Knowledge Acquisition and Scaling Laws
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a theoretical Syntax-Knowledge generative model (Pitman-Yor CRP knowledge plus parametric syntax) viewed through Kolmogorov structure function and two-part coding to derive data and model scaling laws, knowledge-acquisition order, and frequency-based hallucination thresholds, with small-scale experiments.
Problem
Principled explanations for LLM scaling laws, knowledge acquisition dynamics, and factual hallucinations are lacking; the paper seeks a compression-based theory of how LLMs store syntax and knowledge.
Approach
Interprets LLM training as a two-part (data-to-model) code under the Kolmogorov structure function and models data with a hierarchical Syntax-Knowledge model whose knowledge component is a Pitman-Yor process, so Bayesian redundancy yields power-law data and model scaling and frequency-ordered knowledge acquisition.
Key results
- Theory gives data scaling redundancy of Õ(K/N^{1-α}+n_s S/N), and experiments on power-law data show loss following a power law in data size with exponent increasing with skew, while uniform data does not.
- Model scaling theory gives Red_M(C)=Θ(C^{-1/α+1}); experiments show smaller models learn only the most frequent knowledge, and for a 7.2M-parameter model knowledge occurring fewer than 508 times is consistently hallucinated.
- On Fineweb-edu, the inferred knowledge power-law exponent is estimated between 1.35 and 1.5, and PopQA shows a frequency-based knowledge cutoff that shifts lower with model size.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use power-law-distributed rather than uniformly distributed training data | models trained on synthetic data · synthetic data scaling | validation loss scaling | Helped |
| Use uniformly distributed rather than power-law-distributed training data | models trained on synthetic data · synthetic data scaling | validation loss scaling | Hurt |
| Increase skew in the power-law data distribution | models trained on synthetic data · synthetic data scaling | loss decay rate | Helped |
| Train on power-law-distributed rather than uniformly distributed data | models trained on synthetic data · synthetic model scaling | validation loss scaling | Helped |
| Use uniformly distributed rather than power-law-distributed training data | models trained on synthetic data · synthetic model scaling | validation loss scaling | Hurt |
| Increase skew in the power-law data distribution | models trained on synthetic data · synthetic model scaling | loss decay rate | Helped |
| Increase model size | models trained on power-law-distributed data · synthetic data, knowledge frequency classes | validation loss | Helped |
| Increase model size | models trained on power-law-distributed data · synthetic model scaling, knowledge frequency classes | validation loss | Helped |
| Train on additional examples | 7.2M-sized model · synthetic data, knowledge appearing fewer than 508 times | accuracy | No clear effect |
| Increase model size | models pretrained on Fineweb-edu · PopQA | accuracy | Helped |
| Increase training data size | models pretrained on Fineweb-edu · Fineweb-edu | validation loss | Helped |
| Increase model size | models pretrained on Fineweb-edu · Fineweb-edu | validation loss | Helped |
Surprising findings
- Uniform-distribution data yields a loss curve that deviates strongly from power-law scaling with model size, staying flat until a capacity threshold and then dropping sharply.
- Models hallucinate knowledge that occurs many times in training if its frequency falls below a model-size-dependent threshold, regardless of the number of epochs.
Limitations
- Theory relies on simplifying assumptions (Bayesian setting, PYCRP knowledge model, distortion-rate assumptions) and does not analyze transformer architectures or optimization dynamics directly.
- Experiments are small-scale synthetic settings; real-world validation is only qualitative and infers exponents indirectly.
- Focuses only on factual knowledge and does not model relational, procedural, or reasoning knowledge.
Experimental setup
Datasets: synthetic power-law knowledge dataset Fineweb-edu PopQA
Benchmarks: PopQA
Metrics: validation cross-entropy loss accuracy by knowledge frequency class
Baselines: Kaplan et al. scaling laws Jeon and Roy infinite-width network model Maloney et al. random matrix theory scaling
Builds on: Kolmogorov structure function Solomonoff predictor Pitman-Yor process Kaplan scaling laws Hutter data scaling law
Closest papers by meaning
- Can Compressed LLMs Truly Act? An Empirical Evaluation of Agentic Capabilities in LLM Compression (ICML 2025 · similarity 0.90)
- When Less is More: The LLM Scaling Paradox in Context Compression (NeurIPS 2026 · similarity 0.89)
- Preserving LLM Capabilities through Calibration Data Curation: From Analysis to Optimization (NeurIPS 2025 · similarity 0.89)
- Correlation Dimension of Autoregressive Large Language Models (NeurIPS 2025 · similarity 0.89)
- Compressing LLMs: The Truth is Rarely Pure and Never Simple (ICLR 2024 · similarity 0.89)
- LLMs on the Line: Data Determines Loss-to-Loss Scaling Laws (ICML 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.