AI Paper Atlas

Research map / Large language models / Instruction and fine-tuning

Clipping Low-Probability Tokens in SFT Yields a Generalizable Initialization for RL

Tian-Shuo Liu, Chengxing Jia, Haoyu Liu, Pengyuan Wang, Shiyuan Zhang, Jie Fu, Yang Yu

ICML 2026 · Large language models › Instruction and fine-tuning · approach: Fine-tuning for reasoning data

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
clip low-probability tokens in SFT (OPC-SFT) instead of plain SFTSFT cold-start · ScienceWorld+ALFWorld, after GRPOaverage success rate (Qwen2.5-7B): 69.97 -> 77.70Improved
clip low-probability tokens in SFTSFT cold-start · ScienceWorld+ALFWorld, after GRPOaverage success rate (Llama3.2-3B): 79.02 -> 81.49Improved
clip low-probability tokens in SFTSFT cold-start · OOD benchmarks after ALFWorld SFTaverage performance drop: 11.54% smaller dropImproved
clip low-probability tokens in SFTSFT cold-start · ALFWorld/ScienceWorld in-distribution before RLaverage success rate: 64.17 -> 68.61Improved
add NEFTune noise to SFTSFT cold-start · ScienceWorld+ALFWorld, after GRPOaverage success rate (Qwen2.5-7B): 69.97 -> 63.85Hurt
use DFT instead of SFTSFT cold-start · ScienceWorld+ALFWorld, after GRPOaverage success rate: Qwen 7B 69.97 -> 72.84; Llama 79.02 -> 72.57Mixed
apply GRPO to base model without SFT cold-startQwen2.5-7B-Instruct · ScienceWorld+ALFWorldaverage success rate: 77.70 -> 52.86Hurt
clip low-probability tokens in SFTSFT on math (DeepScaleR) data · math reasoningaccuracyNo clear effect
remove clipping but keep ratio-based reweighting (ablation)OPC-SFT · ALFWorld OODgeneralizationHurt
increase old-policy update interval to 1024+ samples (ablation)OPC-SFT · Llama3.2-3B ALFWorld-unseensuccess rate: 77.6 -> 70.9Hurt
update old policy too frequently (interval 64) (ablation)OPC-SFT · Llama3.2-3B ALFWorld-unseensuccess rate: 77.6 -> 75.4Hurt
vary clipping ratio eps in moderate range (ablation)OPC-SFT · agentic and OOD benchmarkssuccess rateNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy