Research map / Large language models / Reasoning and chain of thought

Effective Reasoning Chains Reduce Intrinsic Dimensionality

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Chain-of-thought training and analysis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes intrinsic dimensionality (minimum LoRA parameters to reach a common accuracy threshold) as a quantitative measure of reasoning-chain effectiveness, showing it strongly anti-correlates with generalization across 14 CoT strategies on GSM8K with Gemma-3 1B and 4B.

Problem

Existing explanations of why chain-of-thought helps generalization are qualitative or rely on conflicting quantitative proxies (length, perplexity, KL divergence); there is no principled measure predicting which reasoning strategy generalizes best.

Approach

Fix the model and vary the reasoning strategy used as training targets; measure intrinsic dimensionality as the minimum LoRA parameter count reaching a shared accuracy threshold, and treat lower intrinsic dimension as a compressibility signal that predicts generalization better than length or likelihood-based baselines.

Key results

• On Gemma-3 4B, intrinsic dimensionality achieves Spearman 0.93 with overall accuracy across 14 strategies, versus 0.82 for token perplexity, 0.31 for length, and -0.17 for KL divergence.
• On Gemma-3 1B, intrinsic dimensionality reaches Spearman 0.75 with overall accuracy, versus 0.63 for token perplexity.
• Executed PoT has the lowest intrinsic dimensionality among GSM8K strategies on 4B (1.49M) and the best OOD accuracy (43.40%).

Surprising findings

• On Reasoning Gym cognitive tasks, No CoT has the lowest intrinsic dimensionality and best overall performance, contradicting the benefit of CoT.
• Larger 4B model shows higher intrinsic dimensionality than 1B for ineffective strategies such as No CoT and Very Short CoT.

Limitations

• Computing intrinsic dimensionality requires training many LoRA adapters of varying sizes, making it expensive to optimize directly.
• Evaluated only on Gemma-3 models (1B and 4B) and mainly on GSM8K; generalization beyond this is shown only on a small Reasoning Gym subset.
• Token-level likelihood for the teacher is assumed to be 1 in the perplexity baseline, which simplifies that comparison.

Experimental setup

Datasets: GSM8K GSM-Symbolic GSM-IC GSM-Hard Reasoning Gym

Benchmarks: GSM8K GSM-Symbolic GSM-IC GSM-Hard Reasoning Gym

Metrics: accuracy Spearman rank correlation intrinsic dimensionality (M parameters)

Baselines: token perplexity KL divergence trajectory length LongPPL

Builds on: intrinsic dimensionality (Li et al. 2018) LoRA minimum description length Program-of-Thoughts Plan-and-Solve prompting

Closest papers by meaning

• Deep Hidden Cognition Facilitates Reliable Chain-of-Thought Reasoning (AAAI 2026 · similarity 0.90)
• Unlocking the Capabilities of Thought: A Reasoning Boundary Framework to Quantify and Optimize Chain-of-Thought (NeurIPS 2024 · similarity 0.89)
• When More is Less: Understanding Chain-of-Thought Length in LLMs (ICLR 2026 · similarity 0.89)
• Understanding Chain-of-Thought in LLMs through Information Theory (ICML 2025 · similarity 0.89)
• SCOUT: Teaching Pre-trained Language Models to Enhance Reasoning via Flow Chain-of-Thought (NeurIPS 2025 · similarity 0.88)
• How Does Chain of Thought Think? Mechanistic Interpretability of Chain-of-Thought Reasoning with Sparse Autoencoding (AAAI 2026 · similarity 0.88)