AI Paper Atlas

Research map / Learning theory and algorithms / Kernels and regression theory

Why Less is More (Sometimes): A Theory of Data Curation

Elvis Dohmatob, Mohammad Pezeshki, Reyhane Askari-Hemmat

ICLR 2026 · Learning theory and algorithms › Kernels and regression theory · approach: Statistical perspectives on generalization · 8 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
curate the training set down to 1k hard examples instead of using the 114k OpenThinker setQwen2.5-32B · AIME 2024 (average)Pass@1 (%): 50.2 -> 56.7Improved
curate 59k s1 pool down to 1k examplesQwen2.5-32B · AIME 2024 (average)Pass@1 (%): 53.3 -> 56.7Improved
scale training data from 1k to 1M examplesQwen2.5-32B · AIME hard-level questionsAvg@8 (%): 28.4 -> 64.9Improved
scale from 10k examples to 114k OpenThinkerQwen2.5-32B · AIME hard-level questionsAvg@8 (%): 52.1 -> 47.9Hurt
keep-hard pruning versus random pruning with abundant data and strong generatorsynthetic linear model · Synthetic 2x2 regimestest errorImproved
prune data (keep hard) with small nsynthetic linear model · Synthetic 2x2 regimes (small n)test errorHurt
prune data (keep hard) with a poor generatorsynthetic linear model · Synthetic 2x2 regimes (poor generator, large n)test errorHurt
use keep-easy instead of keep-hard pruning when generator trained on only 160K examplesViT-B/16 pseudo-labeling · ImageNet-1K val (generator n=160K)top-1 accuracyImproved
use keep-hard pruning when generator trained on 1.2M examplesViT-B/16 pseudo-labeling · ImageNet-1K val (generator n=1.2M)top-1 accuracyImproved
apply keep-hard selection at each round of iterative retraining on pseudo-labelsViT-B/16 iterative pseudo-labeling · ImageNet-1K val (iterative retraining)top-1 accuracyImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy