Research map / Large language models / Reasoning and chain of thought

IAPO: Information-Aware Policy Optimization for Token-Efficient Reasoning

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Test-time compute scaling · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes IAPO, a GRPO variant that adds token-level advantages from early-exit estimates of conditional mutual information between each reasoning token and the final answer, plus an entropy-style exploration term, to reduce reasoning length while keeping accuracy.

Problem

RL post-training (GRPO) makes LLM reasoning verbose; existing token-efficiency methods shape advantages by length or position rather than by how much each token contributes to the correct answer.

Approach

Assigns each token an advantage based on its conditional mutual information with the final answer, estimated by appending an answer-elicitation postfix at prefixes (early-exit), accelerated by KV-cache preloading and chunk-wise forwarding, and adds a token-probability exploration term signed by correctness.

Key results

• On GSM8K with Qwen2.5-7B-Instruct, IAPO reduces Length@16 to 84.03 tokens versus 157.08 for the base model and 147.67 for the best baseline S-GRPO, with R@16 of 1.16e-2 vs 6.63e-3 for S-GRPO.
• On MATH-500 with Qwen2.5-7B-Instruct, IAPO gives P@32 of 0.8047 and L@32 of 302 tokens, against DAPO's 0.8148 at 331 tokens; IAPO has the best R@32 among the methods listed for that row.
• The paper reports up to 47% reasoning-length reduction and over 11% wall-clock inference time reduction, but the MATH-500 and DAPO-Math-17k results show many baselines or IAPO are not uniformly best.

Surprising findings

• On DAPO-Math-17k with Qwen2.5-7B-Instruct, IAPO does not beat S-GRPO or GFPO in Ratio@k, and on MATH-500 with the 0.5B model S-GRPO has higher Ratio@k despite lower Pass@k.
• The conclusion cites a 36% token reduction while the abstract and introduction claim up to 47%, an internal inconsistency.

Limitations

• Efficiency gains are not uniform across datasets and model sizes; several baselines beat IAPO on some Ratio@k settings.
• Conditional MI is approximated by answer-entropy differences with a postfix prompt, and the theory relies on a first-order small-step approximation and a monotonicity assumption.
• Experiments use only math and one commonsense dataset with Qwen2.5 base models; the relation to DeepSeek-R1-distilled models is only motivational.

Experimental setup

Datasets: GSM8K MATH-500 DAPO-Math-17k CommonsenseQA

Benchmarks: GSM8K MATH-500 DAPO-Math-17k CommonsenseQA

Metrics: Pass@k Length@k Ratio@k (Pass@k / Length@k) wall-clock inference time

Baselines: DAPO GFPO GTPO S-GRPO GRPO

Builds on: GRPO DAPO GFPO GTPO S-GRPO

Closest papers by meaning

• On the Token Value Inequality in Efficient Reasoning (NeurIPS 2026 · similarity 0.89)
• In-Token Rationality Optimization: Towards Accurate and Concise LLM Reasoning via Self-Feedback (AAAI 2026 · similarity 0.89)
• HAPO: Training Language Models to Reason Concisely via History-Aware Policy Optimization (AAAI 2026 · similarity 0.89)
• Optimizing Anytime Reasoning via Budget Relative Policy Optimization (NeurIPS 2025 · similarity 0.89)
• How Far Are We from Optimal Reasoning Efficiency? (NeurIPS 2025 · similarity 0.89)
• Training-Trajectory-Aware Token Selection (ICML 2026 · similarity 0.89)