AI Paper Atlas

Research map / Probabilistic and scientific modeling / Molecules and proteins

Random Scaling of Emergent Capabilities

Rosie Zhao, Tian Qin, David Alvarez-Melis, Sham M. Kakade, Naomi Saphra

ICML 2026 · Probabilistic and scientific modeling › Molecules and proteins · approach: Sparse concepts and hypothesis testing · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace exact-match accuracy with a continuous max per-token loss metric (ablation)addition length generalization · synthetic additiondistribution across seedsNo clear effect
replace accuracy with negative log likelihood (ablation)Qwen2.5-0.5B · MMLU (10% MMLU mix)distribution across seedsNo clear effect
increase MMLU share of training mix (ablation)Qwen2.5-0.5B · MMLU validationmean accuracy: 1.0% -> 23.0%Improved
scale Qwen2.5-0.5B to Qwen2.5-1.5B at fixed data mix (ablation)Qwen reinitialized · MMLU validationaccuracy across seedsImproved
increase hierarchical-inducing data share (ablation)from-scratch Transformer · question formation OODmean accuracy: 17.2% -> 62.5% (1% to 10%)Improved
use 100% hierarchical-inducing data (ablation)from-scratch Transformer · question formation OODseed variation: mean 74.8%No clear effect
increase depth from 3 to 6 or 8 layers at 90% hierarchical data (ablation)from-scratch Transformer · question formation OODaccuracy distributionMixed
increase batch size from 8 to 64 (ablation)Qwen2.5-1.5B · MMLU continued pretrainingbimodalityHurt
scale width or depth of addition model (ablation)decoder-only Transformer · reverse-order additionW2 distance to largest scaleImproved
increase width with depth fixed at 1 layer (ablation)addition Transformer · reverse-order additionlength generalizationNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy