AI Paper Atlas

Research map / Architectures and efficiency / RNNs and normalization

Pretraining Scaling Laws for Generative Evaluations of Language Models

Rylan Schaeffer, Noam Itzhak Levi, Brando Miranda, Sanmi Koyejo

ICLR 2026 · Architectures and efficiency › RNNs and normalization · approach: Neural scaling laws · 3 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
increase attempts per problem kcompute scaling law · GSM8Kcompute exponent alpha: 1.21e-1 -> 3.75e-1Improved
increase attempts per problem k to about 1e2compute scaling law · GSM8Kirreducible error E0: approx 2 -> 0Improved
increase k to about 3e2parameters+tokens scaling law · GSM8Kirreducible error: 3.87 -> 0Improved
decompose compute into parameters and tokensscaling law fit · GSM8Kin-range fit tightnessImproved
decompose compute into parameters and tokensscaling law prediction · GSM8Kprediction of Pythia-12BNo clear effect
use gold reference likelihood as covariate instead of computescaling law · GSM8Kparameter stability: 2 -> 5 orders of magnitudeImproved
use gold reference likelihood law at large kscaling law prediction · GSM8Krelative errorHurt
move from GSM8K to harder MATHcompute scaling law · MATH vs GSM8Kirreducible error at k=1e4: 0 -> 0.45Hurt
move from GSM8K to harder MATHcompute scaling law · MATH vs GSM8Kcompute exponent at k=1e4: 0.38 -> 0.58Improved
use k=1e4 instead of k=1 or 1e2compute scaling law · GSM8Korders of magnitude needed for extrapolation: 2 -> 1.5Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy