Research map / Large language models / Reasoning and chain of thought

The Tell-Tale Norm: $\ell_2$ Magnitude as a Signal for Reasoning Dynamics in Large Language Models

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Efficient and adaptive thinking

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes the hidden-state l2 norm as a training-free, model-intrinsic signal of reasoning intensity (linked theoretically to SAE reasoning feature activations) and uses it to drive three test-time methods: adaptive layer-wise recursion, endogenous state steering, and l2-guided response selection.

Problem

Layer-wise reasoning dynamics in LLMs are poorly understood, and existing probes (SAE features, output entropy) are either model-specific and expensive to train or retrospective, output-level signals.

Approach

Hidden-state l2 norm rises in late layers during thinking; the paper argues via a (approximation-based) theorem that it bounds the aggregate activation of SAE reasoning features, and uses norm peaks (IQR-based detection) to trigger training-free test-time interventions.

Key results

• l2 norm correlates with SAE reasoning feature activations at Spearman 84-88 across Qwen3-1.7B/8B/14B and LLaMA3-8B, far above entropy-based metrics.
• The three l2-guided methods yield an average 4.51% relative improvement across benchmarks and 9.13% on challenging tasks such as AIME.
• Suppressing high-l2 hidden states in the final 25% of layers of Qwen3-14B degrades performance more than random suppression.

Surprising findings

• Hidden-state norm correlates strongly with SAE reasoning features (Spearman up to ~88) while alternative intrinsic entropy metrics correlate weakly or negatively.

Limitations

• The theoretical bound rests on strong approximating assumptions (near-orthogonality, reconstruction fidelity, semantic orthogonality) that are only empirically sketched.
• The l2 norm is described as a coarse indicator; the paper notes it is not a complete characterization of reasoning and benchmark-dependent norm statistics differ.
• Gains are relative improvements averaged across benchmarks; the paper notes behavior on non-reasoning benchmarks is a failure mode.

Experimental setup

Datasets: MMLU-Pro GSM8K AIME GPQA

Benchmarks: GSM8K AIME GPQA MMLU-Pro

Metrics: accuracy relative improvement Spearman correlation

Baselines: entropy-based signals layer hidden entropy layer output entropy attention entropy random suppression

Builds on: sparse autoencoders chain-of-thought looped transformers IQR outlier peak detection

Closest papers by meaning

• I Have Covered All the Bases Here: Interpreting Reasoning Features in Large Language Models via Sparse Autoencoders (AAAI 2026 · similarity 0.90)
• State of Thought Enables Endogenous Reasoning (NeurIPS 2026 · similarity 0.90)
• Calibrating Reasoning in Language Models with Internal Consistency (NeurIPS 2024 · similarity 0.90)
• ActivationReasoning: Logical Reasoning in Latent Activation Spaces (ICLR 2026 · similarity 0.89)
• Attention Illuminates LLM Reasoning: The Uncovered Preplan-and-Anchor Rhythm Enables Fine-Grained Policy Optimization (ICML 2026 · similarity 0.89)
• Longer Context, Deeper Thinking: Uncovering the Role of Long-Context Ability in Reasoning (NeurIPS 2025 · similarity 0.89)