Research map / Large language models / Language, safety and interpretability
Why Has Predicting Downstream Capabilities of Frontier AI Models with Scale Remained Elusive?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Analyzes why downstream multiple-choice benchmark scores are hard to predict with scale, showing that transformations from per-sample log-likelihoods to accuracy/Brier score progressively degrade score-compute correlation, due to probability mass on incorrect choices.
Problem
Predicting how specific downstream multiple-choice benchmark performance scales with compute remains elusive, even though pretraining loss scaling is well predicted.
Approach
Traces per-sample scores through the transformation chain (log-likelihood → vocab probability → choice-normalized probability → accuracy/Brier score) and shows that metrics depending on incorrect-choice probability mass lose correlation with compute; argues per-choice scaling trends would be needed for prediction.
Key results
- Across benchmarks and families, about 90% of samples show score-compute correlation above 0.75 using log likelihoods, but only about 40% do so for choice-normalized probability of the correct choice.
- The claimed ordering of score-compute correlation (log-likelihood > vocab probability > choice probability > Brier > accuracy) held in at least 82.4% (Pearson), 85.6% (Spearman) and 90.4% (Kendall) of tuples tested.
- Probability masses on correct and incorrect choices show strong positive covariance with compute on multiple benchmarks, suggesting per-choice scaling trends may be feasible.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Transform correct-choice vocabulary probability into choice-normalized probability | Multiple-choice evaluations across model families · ARC Challenge | fraction of samples with score-compute correlation > 0.75: approximately 90% -> 40% | Hurt |
| Transform negative log likelihood into correct-choice vocabulary probability | Multiple-choice evaluations across model families · Multiple-choice benchmarks | Spearman and Kendall score-compute correlation | No clear effect |
| Transform choice-normalized correct-choice probability into Brier Score | Multiple-choice evaluations across model families · Multiple-choice benchmarks | score-compute correlation | No clear effect |
| Transform choice-normalized correct-choice probability into Accuracy | Multiple-choice evaluations across model families · Multiple-choice benchmarks | score-compute correlation | Hurt |
| Use choice-normalized probability rather than correct-choice vocabulary probability | Multiple-choice evaluations across model families · Multiple-choice benchmarks | score-compute correlation | Hurt |
| Use Brier Score rather than choice-normalized correct-choice probability | Multiple-choice evaluations across model families · Multiple-choice benchmarks | score-compute correlation | No clear effect |
| Use Accuracy rather than Brier Score | Multiple-choice evaluations across model families · Multiple-choice benchmarks | score-compute correlation | Hurt |
Surprising findings
- Brier Score, despite being a continuous metric, does not recover the predictability lost when moving to choice-normalized probabilities.
- Probability mass fluctuations on incorrect choices do not average out, so aggregate accuracy cannot be predicted by averaging.
Limitations
- Restricted to log-likelihood-based multiple-choice benchmarks; generative evaluations not studied.
- Analyses assume access to full model family scores across scales and do not perform backtesting or a priori prediction.
- Per-choice scaling trends are only shown to be plausible, not fit and extrapolated.
Experimental setup
Datasets: ARC Easy ARC Challenge HellaSwag MathQA MCTACO MMLU OpenbookQA PIQA RACE SciQ SIQA WinoGrande
Benchmarks: ARC Easy ARC Challenge HellaSwag MathQA MCTACO MMLU OpenbookQA PIQA RACE SciQ SIQA WinoGrande
Metrics: Accuracy Brier Score probability mass on correct choice Pearson correlation Spearman correlation Kendall correlation Wasserstein distance
Builds on: neural scaling laws emergent abilities EleutherAI LM Evaluation Harness
Closest papers by meaning
- Language models scale reliably with over-training and on downstream tasks (ICLR 2025 · similarity 0.88)
- Prescriptive Scaling Reveals the Evolution of Language Model Capabilities (ICML 2026 · similarity 0.88)
- Efficient Prediction of Pass@k Scaling in Large Language Models (NeurIPS 2026 · similarity 0.88)
- The Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMs (ICLR 2026 · similarity 0.87)
- Item Response Scaling Laws: A Measurement Theory Approach for Efficient and Generalizable Neural Scaling Estimation (ICML 2026 · similarity 0.86)
- A Bayesian Model Selection Criterion for Selecting Pretraining Checkpoints (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.