Research map / Large language models / Reasoning and chain of thought

TGRL: Temperature-Grouped Reinforcement Learning for Efficient Exploration in LLMs

NeurIPS 2026 · Large language models › Reasoning and chain of thought · approach: RL with verifiable rewards for reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes TGRL, an RLVR method that splits each rollout group into low- and high-temperature subsets, uses their reward gap as a prompt-level exploration-gain signal, and allocates it to high-temperature tokens via JS-divergence-based weights.

Problem

Exploration is a bottleneck in RLVR: temperature control and test-time scaling either increase rollout budget or leave the benefit of exploration unquantified.

Approach

Turns temperature-induced diversity into a training signal: the mean-reward gap between low- and high-temperature rollouts of the same prompt is injected into a mixed-group advantage, and token-level credit on high-temperature rollouts is weighted by Jensen-Shannon divergence between temperature-scaled next-token distributions from the same logits.

Key results

• On Qwen3-32B, TGRL reaches 70.2% six-benchmark math average vs. 68.6% for the best baseline (DAPO).
• On Qwen3-4B code, CodeForces rating rises from 1377.6 (PPO, best baseline) to 1574.3 (+196.7), and LiveCodeBench Pass@16 reaches 62.1.
• TGRL reaches equivalent accuracy up to 36% faster than DAPO (6.4 h vs 10.0 h) and achieves 86.7% ALFWorld success and 74.2% WebShop success on Qwen2.5-7B-Instruct.

Surprising findings

• Simply raising the sampling temperature of GRPO gives only small gains (0.5% at 14B, 1.1% at 32B), while the mixed-temperature grouping plus JS credit yields larger gains.

Limitations

• Gains on some individual benchmarks are not uniform (e.g., Minerva and AMC23 slightly below baselines at 14B).
• Competition benchmarks have few problems, so reported differences are sensitive to variance (authors add 5-seed checks).

Experimental setup

Datasets: DeepScaleR DeepCoder WebShop ALFWorld

Benchmarks: AIME24 AIME25 AMC23 MATH500 Minerva Olympiad LiveCodeBench CodeForces HumanEval+ ALFWorld WebShop

Metrics: Avg@16 Pass@16 Elo rating success rate task score

Baselines: PPO GRPO DAPO Dr.GRPO RLOO TAMPO EMPG ReAct GRPO+Ent

Builds on: GRPO PPO DAPO Dr.GRPO RLOO

Closest papers by meaning

• LLM-Explorer: A Plug-in Reinforcement Learning Policy Exploration Enhancement Driven by Large Language Models (NeurIPS 2025 · similarity 0.90)
• TempSamp-R1: Effective Temporal Sampling with Reinforcement Fine-Tuning for Video LLMs (NeurIPS 2025 · similarity 0.89)
• Group-in-Group Policy Optimization for LLM Agent Training (NeurIPS 2025 · similarity 0.89)
• EVOLvE: Evaluating and Optimizing LLMs For In-Context Exploration (ICML 2025 · similarity 0.89)
• Expanding LLM Agent Boundaries with Strategy-Guided Exploration (NeurIPS 2026 · similarity 0.89)
• Prompted Policy Search: Reinforcement Learning through Linguistic and Numerical Reasoning in LLMs (NeurIPS 2025 · similarity 0.88)