Research map / Large language models / Reasoning and chain of thought

Characterizing, Evaluating, and Optimizing Complex Reasoning

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Efficient and adaptive thinking · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes the ME^2 principle for reasoning-trace quality, models traces as DAGs for pairwise LLM evaluation, trains a Thinking Reward Model (TRM) on verified-correct trace pairs, and uses it for best-of-N selection and as a shaping reward in GRPO.

Problem

Existing work lacks a unified definition of high-quality reasoning traces, a reliable way to evaluate long, implicitly structured traces, and a scalable way to use such signals to optimize reasoning; PRMs rely on absolute step scoring and ORMs target answer outcomes.

Approach

Define reasoning quality along macro/micro levels and efficiency/effectiveness (ME^2), abstract traces as DAGs built by LLM parent selection, perform robust pairwise LLM comparisons on macro and micro abstractions, and train a Bradley-Terry thinking reward model only on verified-correct traces, then combine it with the verifiable reward via gated shaping in GRPO.

Key results

• Best-of-N on AIME24 with Qwen3-8B rises from 44.7% at N=1 to 64.0% at N=16 (gain up to 19.3%).
• TRM reaches 88.6% validation pairwise accuracy versus 62.5% for ReasonFlux-PRM-7B and 46.3% for Qwen2.5-Math-PRM-7B.
• TRM-guided RL improves Llama-3.1-8B-Instruct by 3.9% on STEM and 3.8% on math over the verifier-only baseline.

Surprising findings

• Qwen2.5-Math-PRM-7B performs near random (46.3%) on pairwise reasoning-trace preference, below chance.
• Direct prompt-based DeepSeek-V3.2 judging produces many ties (232 of 1500), which the DAG abstraction is argued to reduce.

Limitations

• Preference labels come from an LLM judge (DeepSeek-V3.2) rather than humans, with ties excluded for reliability.
• Evaluated on a limited set of models (Qwen2.5-Math, Llama-3.1-8B) for RL training.

Experimental setup

Datasets: WebInstruct-verified

Benchmarks: GPQA-Diamond SuperGPQA MMLU-Pro BBEH FrontierScience AMC AIME24 AIME25 MATH500 OlympiadBench

Metrics: accuracy AVG@32 pairwise win rate

Baselines: ReasonFlux-PRM-7B Qwen2.5-Math-PRM-7B PromptOnly Verifier-only GRPO

Builds on: GRPO Bradley-Terry reward model RLVR ReasonFlux-PRM

Closest papers by meaning

• The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity (NeurIPS 2025 · similarity 0.91)
• How Far Are We from Optimal Reasoning Efficiency? (NeurIPS 2025 · similarity 0.89)
• Don’t Think Longer, Think Wisely: Optimizing Thinking Dynamics for Large Reasoning Models (NeurIPS 2025 · similarity 0.89)
• On Reasoning Strength Planning in Large Reasoning Models (NeurIPS 2025 · similarity 0.89)
• Measuring the Faithfulness of Thinking Drafts in Large Reasoning Models (NeurIPS 2025 · similarity 0.89)
• Tracing the Traces: Latent Temporal Signals for Efficient and Accurate Reasoning (ICLR 2026 · similarity 0.89)