AI Paper Atlas

Research map / Reinforcement learning / RL from feedback

Experience is the Best Teacher: Motivating Effective Exploration in Reinforcement Learning for LLMs

Wenjian Zhang, Kongcheng Zhang, Jiaxin Qi, Jianqiang Huang, Baisheng Lai

ICML 2026 · Reinforcement learning › RL from feedback · approach: RL for LLM reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add hindsight experience (original response plus unmet rubrics) in-context, with loss masking on the experience tokens (ablation)RLVR on Qwen2.5-7B-Instruct · IFBenchaccuracy: 31.6 -> 36.7Improved
add bonus reward on top of hindsight experience (ablation)HeRL on Qwen2.5-7B-Instruct · IFBenchaccuracy: 36.7 -> 39.7Improved
add bonus reward on top of hindsight experience (ablation)HeRL on Llama-3.2-3B-Instruct · WritingBenchscore: 41.1 -> 45.4Improved
use naive hindsight experience (train on revised responses without the experience in context) instead of RLVR baseline (ablation)RLVR on Qwen2.5-7B-Instruct · WritingBenchscore: 54.8 -> 52.7Hurt
use naive hindsight experience instead of RLVR baseline (ablation)RLVR on Qwen3-4B-Instruct-2507 · WritingBenchscore: 83.9 -> 66.2Hurt
use naive hindsight experience instead of RLVR baseline (ablation)RLVR on Llama-3.2-3B-Instruct · HealthBench-500score: 17.8 -> 23.9Improved
use 2 hindsight-guided trajectories per group instead of 1 (ablation)HeRL on Qwen2.5-7B-Instruct · IFBenchaccuracy: 39.7 -> 38.5Hurt
HeRL training instead of RLVRQwen2.5-7B-Instruct · MulDimIFaccuracy: 73.5 -> 83.4Improved
HeRL training instead of RLVR, evaluated on out-of-distribution benchmarksLlama-3.2-3B-Instruct · MATH-500accuracy: 39.6 -> 36.6Hurt
HeRL training instead of RLVR, evaluated on out-of-distribution benchmarksQwen3-4B-Instruct-2507 · MATH-500accuracy: 88.0 -> 89.8Improved
guided sampling by hindsight experience instead of stochastic sampling or entropy-based searchQwen2.5-7B-Instruct · HealthBench (500 questions)PassRate: 28.4 -> 38.2 at 2 attempts (guided vs first-attempt level)Improved
entropy-based token branching search instead of stochastic sampling (ablation)Qwen2.5-7B-Instruct · HealthBench (500 questions)PassRateHurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy