Research map / Large language models / Reasoning and chain of thought

CoT Vectors: Transferring and Probing the Reasoning Mechanisms of LLMs

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Chain-of-thought training and analysis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces CoT Vectors (extracted activation-gap and teacher-student learnable vectors) that are added to a single layer's activations to improve zero-shot CoT reasoning on math and MMLU-Pro, matching LoRA with far fewer parameters.

Problem

Chain-of-thought reasoning improvements via ICL or fine-tuning are costly; the paper asks whether the task-vector paradigm can transfer compact CoT reasoning knowledge into LLMs efficiently, and how such vectors reveal the layer-wise organization of multi-step reasoning.

Approach

Extends task vectors to multi-step reasoning: extracted CoT vectors are averaged activation gaps between CoT and non-CoT inputs; learnable CoT vectors are trained with a frozen-model teacher-student setup (teacher sees CoT, student sees only question plus a learned shift) using CE plus representation alignment (KL) losses, and injected into one layer at inference.

Key results

• Learnable CoT Vector (3.6K params) reaches 83.5% on GSM8K for Qwen2.5-Math-7B vs 74.6% baseline and 79.0% LoRA (10M params), averaging 60.4 vs 56.4 baseline.
• Extracted CoT vectors improve average accuracy by 2.4 (Qwen) and 1.1 (LLaMA) points, but layer-wise performance is unstable with a U-shaped pattern and falls below baseline in middle layers.
• On LLaMA3.1-8B-Instruct, learnable vector averages 56.2 vs 54.6 baseline and 56.0 for LoRA (13.6M params).

Surprising findings

• Extracted CoT vectors injected in middle layers often degrade performance, contrary to prior task-vector findings where middle layers work best.
• LoRA gives limited gains on instruction-tuned models that already have CoT priors, as LoRA loss starts very low.

Limitations

• Optimal injection layer for extracted vectors varies by task and model, requiring layer search with ground truth unavailable in practice.
• Learnable vectors overfit and degrade when applied to middle or deep layers, needing early stopping or reduced learning rates.
• Evaluated on only two models and three datasets with ~1,000 test samples each.

Experimental setup

Datasets: GSM8K MATH MMLU-Pro

Benchmarks: GSM8K MATH-Easy MATH-Hard MMLU-Pro

Metrics: accuracy

Baselines: zero-shot CoT extracted CoT vector LoRA raw activation extracted CoT vector

Builds on: task vectors chain-of-thought prompting LoRA

Closest papers by meaning

• SCOUT: Teaching Pre-trained Language Models to Enhance Reasoning via Flow Chain-of-Thought (NeurIPS 2025 · similarity 0.93)
• To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning (ICLR 2025 · similarity 0.92)
• Enhancing Chain of Thought Prompting in Large Language Models via Reasoning Patterns (AAAI 2025 · similarity 0.91)
• Chain-of-Thought Reasoning Without Prompting (NeurIPS 2024 · similarity 0.91)
• SIM-CoT: Supervised Implicit Chain-of-Thought (ICLR 2026 · similarity 0.91)
• The CoT Encyclopedia: Analyzing, Predicting, and Controlling how a Reasoning Model will Think (ICLR 2026 · similarity 0.91)