Research map / Large language models / Reasoning and chain of thought

Reasoning Structure of Large Language Models

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Efficient and adaptive thinking · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces a puzzle benchmark and an LLM-based pipeline that converts reasoning traces into verifiable claim graphs, plus a flow-based efficiency metric η, to analyze reasoning structure beyond accuracy and token count.

Problem

Final-answer accuracy and token count hide differences in how large reasoning models structure their reasoning; there is no verifiable, structural measure of reasoning over traces.

Approach

Extracts claim-and-dependency graphs from free-form reasoning traces, verifies claim nodes deterministically against a puzzle environment, and models logical flow as an absorbing Markov chain whose structural entropy yields an efficiency score η relative to the minimal solution claim set.

Key results

• GPT-5 accuracy falls from 83.8% on Trivial to 5.7% on Human hard, while using about 4k to 20k tokens.
• Reasoning-flow efficiency η is essentially uncorrelated with token count (r=-0.05, p=0.64) but correlates with accuracy (r=+0.368 pooled in structural metric table).
• Failed Tents traces have about 57% lower η than matched correct traces.

Surprising findings

• Token count correlates strongly with verification overhead rather than solution quality, so extra tokens do not improve reasoning structure.
• Kimi K2 uses the largest token budgets but does not outperform GPT-5 on accuracy.

Limitations

• Claim and rule extraction rely on LLM components, so graphs are not fully deterministic; edge validation is manual on 200 samples with 75.5% fully correct.
• Each puzzle family requires domain-specific claim and rule types, limiting the verifier to puzzle settings.
• Failed-trace analysis is limited to Tents with only 10 incorrect traces.

Experimental setup

Datasets: 21 grid logic puzzles from Simon Tatham's puzzle collection

Benchmarks: PUZZLE benchmark

Metrics: accuracy token count reasoning-flow efficiency η structural entropy claim accuracy first-error depth

Baselines: GPT-5 Qwen3 235B DeepSeek V3.2 Kimi K2

Builds on: Chain-of-Thought Tree-of-Thoughts Graph-of-Thoughts Simon Tatham's puzzle collection ZebraLogic

Closest papers by meaning

• When Reasoning Meets Its Laws (NeurIPS 2026 · similarity 0.91)
• The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity (NeurIPS 2025 · similarity 0.90)
• Exposing Weaknesses of Large Reasoning Models through Graph Algorithm Problems (ICLR 2026 · similarity 0.90)
• ZebraLogic: On the Scaling Limits of LLMs for Logical Reasoning (ICML 2025 · similarity 0.90)
• Beyond Semantics: The Unreasonable Effectiveness of Reasonless Intermediate Tokens (NeurIPS 2026 · similarity 0.90)
• Mitigating Overthinking in Large Reasoning Language Models via Reasoning Path Deviation Monitoring (NeurIPS 2026 · similarity 0.89)