AI Paper Atlas

Research map / Reinforcement learning / RL from feedback

PretrainZero: Reinforcement Active Learning on Pretraining Data

Xingrun Xing, Zhiyuan Fan, Jie Lou, Guoqi Li, Jiajun Zhang, Debing Zhang

ICML 2026 · Reinforcement learning › RL from feedback · approach: Scaling RL with verifiable rewards

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add reinforcement active learning (learned mask generation) instead of random span masks in RLPT (ablation)PretrainZero · Qwen3-4B-Base overall averageoverall average: 35.41 -> 39.61Improved
replace continued next-token pretraining on Wikipedia with PretrainZeroQwen3-4B-Base · overall averageoverall average: 15.89 -> 39.61Improved
supervised fine-tuning of masked-span QA pairs on Wikipedia instead of RLQwen3-4B-Base · overall averageoverall average: 32.36 -> 24.27Hurt
add active learning RLPT (PretrainZero) on Qwen3-8B-Base (ablation)Qwen3-8B-Base · overall averageoverall average: 40.96 -> 42.78Improved
add active learning RLPT (PretrainZero) on Qwen3-30B-A3B-MoE-Base (ablation)Qwen3-30B-A3B-MoE-Base · overall averageoverall average: 40.38 -> 43.55Improved
add active learning RLPT (PretrainZero) on SmolLM3-3B-Base (ablation)SmolLM3-3B-Base · overall averageoverall average: 20.25 -> 23.41Improved
select high-entropy tokens instead of random tokens for next-token RLPT on Wikipedia (ablation)Random NTP RLPT · MMLU-ProaccuracyHurt
use random masked span instead of random next-token prediction in RLPT (ablation)Random NTP RLPT · MMLU-ProaccuracyHurt
add PretrainZero RLPT before general RLVR post-training (2000 RLPT steps) (ablation)Qwen3-4B-Base · overall average after RLVRoverall average: 37.90 -> 40.46Improved
increase RLPT steps from 1000 to 2000 before RLVR (ablation)PretrainZero · overall average after RLVRoverall average: 39.15 -> 40.46Improved
use Wikipedia instead of MathPile as RLPT data (ablation)PretrainZero · MMLU-ProaccuracyImproved
add PretrainZero RLPT instead of Random RLPT before RLVR (ablation)Qwen3-4B-Base · MMLU-Pro after RLVRaccuracy: 58.62 -> 61.15Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy