AI Paper Atlas

Research map / Large language models / Instruction and fine-tuning

On the Entropy Dynamics in Reinforcement Fine-Tuning of Large Language Models

Shumin Wang, Yuexiang Xie, Wenhao Zhang, Yuchang Sun, Yanxi Chen, Yaliang Li, Yanyong Zhang

ICML 2026 · Large language models › Instruction and fine-tuning · approach: Fine-tuning generalization and delta compression · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
apply ClipB (batch-normalized entropy-discriminator clipping) on top of GRPOGRPO, Qwen2.5-7B-Instruct · Average over 5 math benchmarksAvg@K: 39.36 -> 41.38Improved
apply ClipV (vocabulary-normalized clipping) on top of GRPOGRPO, Qwen2.5-7B-Instruct · Average over 5 math benchmarksAvg@K: 39.36 -> 40.61Improved
apply ClipV on top of GRPOGRPO, Qwen2.5-14B-Instruct · Average over 5 math benchmarksAvg@K: 45.57 -> 49.44Improved
apply ClipB on top of GRPOGRPO, Qwen2.5-14B-Instruct · Average over 5 math benchmarksAvg@K: 45.57 -> 48.81Improved
apply ClipB on top of GRPOGRPO, Qwen2.5-7B-Instruct · DAPO500Pass@8: 76.8 -> 80.2Improved
apply ClipB on top of GRPOGRPO, Qwen2.5-7B-Instruct · AIME24Pass@32: 50.00 -> 56.67Improved
compare other entropy methods (Forking Tokens, Rewarding Unlikely) against GRPOGRPO, Qwen2.5-7B-Instruct · Average over 5 math benchmarksAvg@K: 39.36 -> 38.33 (Forking Tokens), 38.78 (Rewarding Unlikely)Hurt
apply Clip-Cov or KL-Cov on top of GRPOGRPO, Qwen2.5-7B-Instruct · Average over 5 math benchmarksAvg@K: 39.36 -> 39.37 (Clip-Cov), 39.54 (KL-Cov)No clear effect
retain only updates for tokens with S*>0 in positive samples (ablation)GRPO training · Entropy dynamics, DAPO-Math-17kentropyHurt
mask gradients of S*>0 tokens in positive samples (ablation)GRPO training · Entropy dynamics, DAPO-Math-17kentropyMixed
increase mu threshold in ClipB/ClipV (ablation)ClipB, ClipV · Clip fraction, DAPO-Math-17kclip fractionImproved
apply ClipB to GRPOGRPO, Qwen2.5-7B-Instruct · Pass-rate distribution, test setshare of all-fail questions (%): 66.7 -> 63.3Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy