Research map / Large language models / Reasoning and chain of thought

Boosting LLM Reasoning via Human-Inspired Reward Shaping

NeurIPS 2026 · Large language models › Reasoning and chain of thought · approach: RL with verifiable rewards for reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes T2T, a sequence-level RLVR reward that thickens (rewards length) on incorrect attempts and thins (penalizes length) on correct ones, scaled by on-policy pass rate, and reports gains over GRPO and length/gradient-reweighting baselines on math benchmarks.

Problem

Standard RLVR uses binary correctness rewards that entangle exploration on unmastered problems with condensation on mastered ones, and provides little gradient when all samples are incorrect or all correct.

Approach

A difficulty-aware reward that scales a length score by the group pass rate: incorrect outputs get a bonus for length (thickening) proportional to (1 - pass rate), and correct outputs get a penalty for length (thinning) proportional to pass rate, with correctness always dominating since alpha is below 0.5.</ml:parameter>

Key results

• On Qwen3-14B, T2T improves AIME'25 Pass@64 from 51.5 (GRPO) to 57.9 and AMC'23 Pass@64 from 94.6 to 98.4.
• On Qwen3-4B, AIME'24 Pass@64 rises from 48.6 (GRPO) to 56.3 for T2T.
• On Llama-3.2-3B-Instruct, AMC'23 Pass@64 goes from 43.7 (GRPO) to 49.1 (T2T).

Surprising findings

• On the controlled comparison, T2T shortens correct responses relative to GRPO even though the paper frames length as merely an instrument.
• Ablation shows thinning-only performs worse than thickening-only on several benchmarks, suggesting exploration reward matters more than compression alone.

Limitations

• Baselines for Llama-3.2-3B-Instruct and Qwen3-14B include only GRPO due to computational constraints.
• Gains on Pass@1 are often small or absent (e.g., MATH-500 on DeepSeek-R1-Distill-Qwen-1.5B and Qwen3-4B), and Pass@64 gains are reported for most settings.
• Theoretical analysis shows the update does not correspond to the gradient of a fixed expected reward; the objective is only partly characterized.

Experimental setup

Datasets: DAPO-MATH-17k BBH HumanEval

Benchmarks: MATH-500 AIME'24 AIME'25 AMC'23 BBH HumanEval

Metrics: Pass@1 Pass@64 Pass@k average response length 3-shot exact match pass@k on HumanEval

Baselines: GRPO LASER W-REINFORCE EntroPIC

Builds on: GRPO DAPO Verl LASER W-REINFORCE

Closest papers by meaning

• $\textbf{Re}^{2}$: Unlocking LLM Reasoning via Reinforcement Learning with Re-solving (ICLR 2026 · similarity 0.91)
• Rectifying LLM Thought from Lens of Optimization (ICLR 2026 · similarity 0.90)
• Exploration vs Exploitation: Rethinking RLVR through Clipping, Entropy, and Spurious Reward (ICLR 2026 · similarity 0.90)
• Reshaping Reasoning in LLMs: A Theoretical Analysis of RL Training Dynamics through Pattern Selection (ICLR 2026 · similarity 0.90)
• A Simple "Motivation" Can Enhance Reinforcement Finetuning of Large Reasoning Models (ICLR 2026 · similarity 0.90)
• Prioritize the Process, Not Just the Outcome: Rewarding Latent Thought Trajectories Improves Reasoning in Looped Language Models (ICML 2026 · similarity 0.90)