Research map / Graph learning / Graph self-supervised learning

Topology of Reasoning: Understanding Large Reasoning Models through Reasoning Graph Properties

NeurIPS 2025 · Graph learning › Graph self-supervised learning · approach: LLMs and graph foundation models · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces reasoning graphs built by k-means clustering of segment hidden states and analyzes cycles, diameter, and small-world index, finding distilled DeepSeek-R1 Qwen models show more cycles, larger diameters, and higher small-world index than base Qwen2.5 models, and that s1-v1.1 SFT data increases graph diameter.

Problem

The internal mechanisms behind the reasoning success of large reasoning models are poorly understood; the paper asks how graph-theoretic properties of their internal state trajectories differ from base models and relate to accuracy and SFT data quality.

Approach

Extract a reasoning graph by clustering per-step hidden-state representations (k-means, K=200) into nodes and connecting consecutive visited nodes as edges, then measure cycle presence/count, diameter, and small-world index to compare reasoning models against base models and across SFT datasets.

Key results

• DeepSeek-R1-Distill-Qwen-32B shows about 5 cycles per sample on average and larger graph diameters than Qwen2.5-32B across layers on GSM8K, MATH500, and AIME 2024.
• Cycle detection ratio peaks at 100% for the 14B distilled model, while the 32B model (highest accuracy) has the largest diameter and cycle count; the 14B model's extra cycles are attributed to language mixing.
• Training Qwen2.5-32B-Instruct with s1-v1.1 instead of s1-v1.0 yields larger reasoning graph diameters across layers at 200 and 400 steps, tracking higher AIME 2024 accuracy (56.7% vs 50.0%).

Measured effects reported in the paper

Surprising findings

• The 14B distilled model has a 100% cycle detection ratio yet underperforms the 32B model, with cycles attributed to language mixing.
• Intermediate layers show lower cycle detection rates than early and late layers.

Limitations

• Analysis is correlational; it does not establish that graph properties cause reasoning gains.
• Does not provide feature-level or circuit-level mechanistic analysis.
• Only limited SFT experiments (s1 datasets) connect graph properties to training; concrete guidelines for model construction remain insufficient.

Experimental setup

Datasets: GSM8K MATH500 AIME 2024 s1K v1.0 s1K-1.1 LIMO StrategyQA BIG-Bench LogicalDeduction

Benchmarks: GSM8K MATH500 AIME 2024 StrategyQA LogicalDeduction

Metrics: cycle detection ratio cycle count graph diameter clustering coefficient average path length small-world index accuracy

Baselines: Qwen2.5-32B Qwen2.5-Math-1.5B Qwen2.5-Math-7B Llama-3.1-8B LIMO

Builds on: reasoning graph framework from prior work on hidden-state clustering s1 Erdős–Rényi random graph baseline Dijkstra's algorithm

Closest papers by meaning

• Exposing Weaknesses of Large Reasoning Models through Graph Algorithm Problems (ICLR 2026 · similarity 0.93)
• Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning (ICLR 2024 · similarity 0.91)
• DAG-Math: Graph-Guided Mathematical Reasoning in LLMs (ICLR 2026 · similarity 0.90)
• Clustering as Reasoning: A $k$-Means Interpretation of Chain-of-Thought Graph Learning (ICML 2026 · similarity 0.90)
• Talk like a Graph: Encoding Graphs for Large Language Models (ICLR 2024 · similarity 0.89)
• $\texttt{G1}$: Teaching LLMs to Reason on Graphs with Reinforcement Learning (NeurIPS 2025 · similarity 0.89)