AI Paper Atlas

Research map / Learning theory and algorithms / Prediction and decision losses

Can Small Training Runs Reliably Guide Data Curation? Rethinking Proxy-Model Practice

Jiachen T. Wang, Tong Wu, Kaifeng Lyu, James Zou, Dawn Song, Ruoxi Jia, Prateek Mittal

ICLR 2026 · Learning theory and algorithms › Prediction and decision losses · approach: Active learning and subset selection · 2 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
train proxy model with tiny learning rate (below 1e-4) instead of standard learning rateproxy models GPT2-Small, Pythia-70M, OPT-125M · Pile validation loss ranking of 23 recipesSpearman rank correlation: < 0.75 -> > 0.92Improved
train proxy with tiny learning rateproxy models · downstream benchmarks and Pile domainsrank correlationImproved
train proxy with tiny learning rateproxy models · top-k decision regret on Pile val lossvalidation loss gap: > 0.25 -> near zeroImproved
increase learning rate slightly from lower to higher valueGPT2-Small on DCLM vs DCLM-dedup-GS · Pile validation loss and 5 downstream tasksdataset rankingHurt
tune proxy learning rate within near-optimal range instead of a fixed oneproxy dataset comparison · DCLM vs stricter dedup variantranking stabilityHurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy