Research map / Large language models / Reasoning and chain of thought

Geometry of Reason: Spectral Signatures of Valid Mathematical Reasoning

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Efficient and adaptive thinking · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a training-free spectral graph method that classifies validity of mathematical proofs from transformer attention Laplacian diagnostics (HFER, Fiedler value, smoothness, entropy) across seven models on MiniF2F.

Problem

Verifying whether a language model's mathematical reasoning is genuinely valid or pattern-matching, without expensive learned verifiers, brittle output heuristics, or reliance on compiler acceptance.

Approach

Treats each attention matrix as a symmetrized weighted token graph and extracts graph-Laplacian/graph-Fourier diagnostics (Fiedler value, HFER, spectral entropy, smoothness) of hidden states as a training-free validity signal; reports that the best metric depends on attention design (SWA shifts HFER to smoothness).

Key results

• Single-threshold spectral classifiers reach Cohen's d up to 3.30 (Phi-3.5-mini) and 85.9-94.9% accuracy across seven models, with p < 10^-47 for all.
• Nested CV accuracy 82.8-85.9% across models with zero trained parameters.
• HFER reranking in Best-of-16 proof search reaches 34.2% Pass@1 vs 29.8% for log-probability on Llama-3.1-8B (MiniF2F), with ensemble at 37.1%.

Surprising findings

• Proofs rejected by Lean for timeouts or missing imports are classified as valid by the spectral method (Platonic validity), confirmed by manual audit with kappa=0.82.
• Sliding Window Attention shifts the best discriminating feature from HFER to smoothness.

Limitations

• Validation scoped to formalized Lean 4 reasoning on MiniF2F; informal chain-of-thought signal is weaker (d=0.78).
• Optimal thresholds are architecture-specific and need per-model calibration.
• Correlation-based diagnostic; mechanistic account of emergence remains open.

Experimental setup

Datasets: MiniF2F MATH

Benchmarks: MiniF2F MATH

Metrics: accuracy Cohen's d Mann-Whitney p-value Welch t-test p-value Pass@1 AUC-ROC

Baselines: Token Entropy reranker Log-Prob reranker Random Forest supervised linear probe hallucination probe on Llama-3.1-8B layer 16 majority class

Builds on: attention graphs graph signal processing induction heads Lanczos iteration

Closest papers by meaning

• Verifying Chain-of-Thought Reasoning via Its Computational Graph (ICLR 2026 · similarity 0.89)
• Topology of Reasoning: Understanding Large Reasoning Models through Reasoning Graph Properties (NeurIPS 2025 · similarity 0.87)
• Putnam-AXIOM: A Functional & Static Benchmark for Measuring Higher Level Mathematical Reasoning in LLMs (ICML 2025 · similarity 0.87)
• Boule or Baguette? A Study on Task Topology, Length Generalization, and the Benefit of Reasoning Trace (NeurIPS 2026 · similarity 0.86)
• Beyond Semantics: The Unreasonable Effectiveness of Reasonless Intermediate Tokens (NeurIPS 2026 · similarity 0.86)
• DAG-Math: Graph-Guided Mathematical Reasoning in LLMs (ICLR 2026 · similarity 0.86)