Research map / Large language models / Reasoning and chain of thought

Verifying Chain-of-Thought Reasoning via Its Computational Graph

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Long CoT dynamics and efficiency · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Circuit-based Reasoning Verification (CRV), which builds transcoder-based attribution graphs of Llama 3.1 8B Instruct CoT steps and classifies step correctness from structural graph features, with causal feature-level interventions to fix errors.

Problem

Existing CoT verifiers are black-box or gray-box and can detect that a reasoning step is likely wrong but cannot explain why the underlying computation fails.

Approach

Treat attribution graphs built over per-layer transcoder features as execution traces of latent circuits; extract fixed-size structural fingerprints (global, node influence/activation, topological features) from each step's pruned graph and train a gradient boosting classifier to predict step correctness, then causally test error features by clamping or amplifying them.

Key results

• On Synthetic Arithmetic, CRV reaches AUROC 92.47 vs. 76.45 for the strongest baseline (Energy).
• On GSM8K, CRV achieves AUROC 70.17 and AUPR 14.3, vs. 62.55 AUROC for Energy.
• CRV trained only on Arithmetic transfers poorly to GSM8K (AUROC 57.04), while a combined-training classifier reaches 90.51 on Arithmetic and 70.62 on GSM8K.

Surprising findings

• Zero-shot cross-domain transfer of the CRV classifier is weak; error signatures appear task-specific.
• Fine-tuning transcoders on instruction-tuning data gave no consistent verification gain.

Limitations

• Zero-shot transfer across reasoning domains is poor, so classifiers must be trained per domain or on a mix.
• Computationally intensive attribution graph construction; not a practical drop-in verifier.
• Only evaluated on one model (Llama 3.1 8B Instruct) and small set of tasks; intervention is a single-case proof-of-concept.

Experimental setup

Datasets: RedPajama-V2 LMSYS-Chat-1M GSM8K

Benchmarks: Synthetic (Boolean) Synthetic (Arithmetic) GSM8K

Metrics: AUROC AUPR FPR@95

Baselines: MaxProb PPL Entropy Temp. Scaling Energy CoE-R CoE-C CoT-Kinetics LR Probe MLP Probe

Builds on: per-layer transcoders circuit tracing attribution graphs Chain-of-Embedding CoT-Kinetics Temperature Scaling

Closest papers by meaning

• Deep Hidden Cognition Facilitates Reliable Chain-of-Thought Reasoning (AAAI 2026 · similarity 0.90)
• Pause and Reflect: Conformal Aggregation for Chain-of-Thought Reasoning (NeurIPS 2026 · similarity 0.89)
• DAG-Math: Graph-Guided Mathematical Reasoning in LLMs (ICLR 2026 · similarity 0.89)
• Unlocking the Capabilities of Thought: A Reasoning Boundary Framework to Quantify and Optimize Chain-of-Thought (NeurIPS 2024 · similarity 0.89)
• FaithCoT-Bench: Benchmarking Instance-Level Faithfulness of Chain-of-Thought Reasoning (ICLR 2026 · similarity 0.89)
• Topology of Reasoning: Understanding Large Reasoning Models through Reasoning Graph Properties (NeurIPS 2025 · similarity 0.89)