Research map / Large language models / Reasoning and chain of thought

DenseSteer: Steering Small Language Models towards Dense Math Reasoning

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Search and self-correction in reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DenseSteer, a training-free inference-time steering method that extracts a steering vector from GPT-5.1-rewritten, denser versions of a small model's own math solutions and injects it into the residual stream to improve small-model math reasoning.

Problem

Small language models (<=3B) underperform on multi-step math reasoning; distillation from larger teachers is costly and suffers from distribution mismatch. The paper asks whether inference-time steering toward a denser reasoning style can close part of this gap without training.

Approach

Identifies "Dense Reasoning" (fewer steps with higher tokens-per-step density) as a trait of stronger Qwen-2.5 models, then builds contrastive pairs by minimally rewriting the small model's own outputs into denser form (Dense-Rewriting), and steers the residual stream at a middle layer with the mean difference vector, keeping NLL low.

Key results

• On Qwen2.5-3B-Instruct GSM8K, DenseSteer reaches 84.8% vs 83.6% zero-shot CoT and 82.2% for Mix-Long distillation.
• On Qwen2.5-3B-Instruct, DenseSteer gets 62.5 sample-weighted average vs 61.2 zero-shot and 61.3 for Mix-Large; AIME 10.0% vs 0.0% zero-shot.
• On Llama-3.2-3B-Instruct, DenseSteer averages 52.7 vs 52.5 zero-shot, so gains are mixed and not consistent across models.

Surprising findings

• Prompt engineering with the dense-style instruction collapses performance (e.g., 20.0% on GSM8K for Qwen2.5-3B), as small models stop producing explicit reasoning steps.
• Llama-3.2-3B DenseSteer gains are negligible or negative on several benchmarks (GSM8K 79.7 vs 79.0 zero-shot, MATH500 43.4 vs 46.0).

Limitations

• Gains are small and mixed across models (e.g., Llama-3.2-3B average 52.7 vs 52.5 for zero-shot; Llama-3.2-1B is only 24.7 vs 24.0 zero-shot on average), and results come from a single greedy run.
• Method only shown to reorganize latent reasoning; cannot add missing knowledge or substantially stronger search; validated mainly on math.
• Dense-Rewriting relies on a commercial rewriter (GPT-5.1) and layer/lambda choices tuned on validation data.

Experimental setup

Datasets: GSM8K MATH-500 AMC 2023 AIME 2024 OlympiadBench LogiQA MMLU BBH HotpotQA

Benchmarks: GSM8K MATH500 AMC OlympiadBench AIME LogiQA MMLU BBH HotpotQA

Metrics: accuracy exact match F1 token-level NLL sample-weighted average accuracy

Baselines: Zero-shot CoT Prompt Engineering Short CoT Long CoT Mix-Long Mix-Large SEAL InFamilySteer

Builds on: steering vectors chain-of-thought knowledge distillation

Closest papers by meaning

• MathFimer: Enhancing Mathematical Reasoning by Expanding Reasoning Steps through Fill-in-the-Middle Task (ICLR 2026 · similarity 0.90)
• MARGE: Improving Math Reasoning with Guided Exploration (ICML 2025 · similarity 0.90)
• UGMathBench: A Diverse and Dynamic Benchmark for Undergraduate-Level Mathematical Reasoning with Large Language Models (ICLR 2025 · similarity 0.89)
• Efficient Reasoning via Constrained Optimization in Latent Space (NeurIPS 2026 · similarity 0.89)
• OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Data (ICLR 2025 · similarity 0.89)
• Putnam-AXIOM: A Functional & Static Benchmark for Measuring Higher Level Mathematical Reasoning in LLMs (ICML 2025 · similarity 0.89)