AI Paper Atlas

Research map / Large language models / Language, safety and interpretability

Learning is Forgetting; LLM Training As Lossy Compression

Henry Conklin, Tom Hosking, Tan Yi-Chern, Jonathan D. Cohen, Sarah-Jane Leslie, Thomas L. Griffiths, Max Bartolo, Seraphina Goldfarb-Tarrant

ICLR 2026 · Large language models › Language, safety and interpretability · approach: LLM pruning and scaling data · 6 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use optimality (expressivity per bit of complexity) as predictor instead of expressivity alone47 open-weights LLMs · aggregate of 6 benchmarksSpearman/Pearson correlation: r=0.08 (expressivity) -> r=0.52 (optimality)Improved
lower token-level complexity47 open-weights LLMs · aggregate of 6 benchmarkscorrelation with performance: r=-0.38Improved
more preference information in representations47 open-weights LLMs · aggregate of 6 benchmarkscorrelation with performance: r=0.76Improved
optimal compression of C4 as predictor for instruction following47 open-weights LLMs · IFEvalcorrelation with performance: r=0.07, p=0.631No clear effect
preference information as predictor of instruction following47 open-weights LLMs · IFEvalcorrelation with performance: r=0.39, p=0.004Improved
smaller model scale (1B vs 7B and 32B OLMo2) (ablation)OLMo2 pre-training · information plane over pre-trainingapproach to IB boundHurt
condition on wider context (token to trigram/quadgram back-off) (ablation)OLMo2 7B · information plane over pre-trainingproximity to IB boundImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy