AI Paper Atlas

Research map / Reinforcement learning / RL from feedback

CDE: Curiosity-Driven Exploration for Efficient Reinforcement Learning in Large Language Models

Runpeng Dai, Linfeng Song, Haolin Liu, Zhenwen Liang, Dian Yu, Haitao Mi, Zhaopeng Tu, Rui Liu, Tong Zheng, Hongtu Zhu, Dong Yu

ICLR 2026 · Reinforcement learning › RL from feedback · approach: Scaling RL with verifiable rewards

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add perplexity-based actor curiosity bonusGRPO on Qwen3-4B-Base · Average over MATH/AMC23/AIME24/AIME25accuracy: 48.2 -> 50.6Improved
add perplexity-based actor curiosity bonusGRPO on Qwen3-4B-Base · AIME24Pass@16: 41.9 -> 48.5Improved
add perplexity bonusPPO on Qwen3-4B-Base · Average over benchmarksaccuracy: 46.5 -> 47.7Improved
use multi-head critic with 4 headsPPO on Qwen3-4B-Base · Average over benchmarksaccuracy: 46.5 -> 48.5Improved
use multi-head critic with 2 heads (ablation)PPO on Qwen3-4B-Base · Average over benchmarksaccuracy: 46.5 -> 46.6No clear effect
increase heads beyond 4 (ablation)multi-head PPO · Average over benchmarksaccuracy: 48.5 (4) vs 48.1 (8) vs 48.6 (16)No clear effect
no weight decay on PPL bonus versus staircase decay (ablation)GRPO with PPL bonus · Average over benchmarksaccuracy: 50.6 (staircase) -> 48.2 (no decay)Hurt
staircase instead of linear or cosine decay (ablation)GRPO with PPL bonus · Average over benchmarksaccuracy: 49.7 (cosine) -> 50.6 (staircase)Improved
sub-sample fraction zeta 0.5 vs 1 (ablation)multi-head PPO (4 and 16 heads) · Average over benchmarksaccuracy: 48.6 vs 48.4 (16 heads); 48.4 vs 48.5 (4 heads)No clear effect
add PPL bonus to GRPOGRPO · entropy during trainingpolicy entropyImproved
add PPL bonus to GRPOGRPO · training calibrationPPL gap between correct and incorrectImproved
replace learned exploration with hash-based count bonus (SimHash)GRPO · preliminary experimenthash cell distributionHurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy