AI Paper Atlas

Research map / Reinforcement learning / RL from feedback

Good SFT Optimizes for SFT, Better SFT Prepares for Reinforcement Learning

Dylan Zhang, Yufeng Xu, Haojin Wang, Qingzhi Chen, Hao Peng

ICML 2026 · Reinforcement learning › RL from feedback · approach: Scaling RL with verifiable rewards · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace SFT with PEAR token-level weighting before GRPOQwen3-1.7B-Base · SynLogicPass@1 (%): 13.1 -> 38.3Improved
replace SFT with PEAR before GRPODeepSeek-Distill-Qwen-1.5B · AIME-25Pass@8 (%): 5 -> 35Improved
replace SFT with PEAR before GRPODeepSeek-Distill-Qwen-1.5B · Math average Pass@1accuracy (%): 14 -> 39Improved
replace SFT with PEAR before GRPOQwen3-8B-Base · AIME-25 Pass@8Pass@8 (%): 35 -> 35No clear effect
replace SFT with PEAR before GRPOQwen2.5-1.5B-Math · AMC23 Pass@8Pass@8 (%): 80 -> 78Hurt
use single-step token importance weight instead of SFTQwen3-1.7B-Base · Math average accuracyaccuracy (%): 23 -> 18Hurt
use suffix-based PEAR weight instead of single-step weight (ablation)Qwen3-8B-Base · Math average accuracyaccuracy (%): 30 -> 41Improved
block size 8 instead of token-level (B=1) (ablation)PEAR · SynLogic Qwen3-1.7B-BasePass@1 (%): 45.00 -> 68.75Improved
block size 4 instead of sequence-level weighting (ablation)PEAR · SynLogic Qwen3-1.7B-BasePass@8 (%): 67.50 -> 83.75Improved
add negative examples to PEAR (ablation)PEAR · SynLogicPass@1Improved
use Qwen3-32B proxy instead of true behavior policy (ablation)PEAR black-box · SynLogic Qwen3-1.7B-BasePass@1No clear effect
widen clip range from [-0.08,0.08] to [-0.30,0.30] (ablation)PEAR · SynLogic Qwen3-1.7B-BasePass@32 (%): 42.50 -> 51.25No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy