Research map / Large language models / Reasoning and chain of thought

I Have Covered All the Bases Here: Interpreting Reasoning Features in Large Language Models via Sparse Autoencoders

AAAI 2026 · Large language models › Reasoning and chain of thought · approach: Search and self-correction in reasoning

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Identifies reasoning-specific sparse autoencoder features in DeepSeek-R1-Llama-8B via a new ReasonScore metric over a reasoning vocabulary, and shows steering them improves benchmark accuracy and lengthens reasoning traces.

Problem

Reasoning LLMs such as DeepSeek-R1 perform complex reasoning, but the internal mechanisms behind reasoning behavior are unexplored; it is unclear whether reasoning is encoded in identifiable internal features.

Approach

Select SAE features whose activations concentrate on a vocabulary of reasoning words (e.g., "but", "alternatively", "perhaps") and their context windows, using a ReasonScore with an entropy penalty; validate them by manual and automatic interpretation, steering, and stage-wise model diffing.

Key results

• Steering feature #4395 raises MATH-500 maj@4 from 93.2 to 95.4 (+2.2%), and feature #16778 lengthens traces by about 20.5% on MATH-500.
• Of 200 ReasonScore features, 46 were manually verified as reflecting uncertainty, exploration, or reflection.
• In stage-wise diffing, 60% of verified reasoning features appear in the S→D→F trajectory versus 0% after switching to the reasoning model alone and 4% after reasoning data alone.

Surprising findings

• Banning reasoning vocabulary words drops AIME 2024 pass@1 from 53.3 to 20.0, while banning random words has little effect.
• Feature #3942 shortens reasoning traces by about 7.7% with negligible performance loss.

Limitations

• ReasonScore depends on hyperparameters (window size, entropy penalty alpha) that were not thoroughly ablated.
• Only 46 of 200 candidate features could be confidently interpreted; other reasoning features may be missed.
• Diffing with cosine threshold 0.7 may yield false negatives and positives, and only 60% of verified features appear in the final stage.

Experimental setup

Datasets: OpenThoughts-114k LMSys-Chat-1M SlimPajama Google Books Ngram Corpus

Benchmarks: AIME 2024 MATH-500 GPQA Diamond

Metrics: maj@4 pass@1 average output tokens feature presence (cosine similarity) L0 sparsity fraction of variance explained

Baselines: No steering

Builds on: Sparse autoencoders (Anthropic April update) stage-wise model diffing Keyphrase extraction windowing

Closest papers by meaning

• Step-Level Sparse Autoencoder for Reasoning Process Interpretation (ICML 2026 · similarity 0.93)
• ActivationReasoning: Logical Reasoning in Latent Activation Spaces (ICLR 2026 · similarity 0.90)
• Automatically Interpreting Millions of Features in Large Language Models (ICML 2025 · similarity 0.90)
• Exploring the Limits of Sub-Billion Language Model Reasoners with Open Training Recipes (ICLR 2026 · similarity 0.90)
• The Tell-Tale Norm: $\ell_2$ Magnitude as a Signal for Reasoning Dynamics in Large Language Models (ICML 2026 · similarity 0.90)
• Are Sparse Autoencoders Useful? A Case Study in Sparse Probing (ICML 2025 · similarity 0.89)