Research map / Large language models / Reasoning and chain of thought

Test-time Diverse Reasoning by Riemannian Activation Steering

AAAI 2026 · Large language models › Reasoning and chain of thought · approach: Efficient and adaptive thinking · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SPREAD, an unsupervised test-time activation steering method that optimizes per-sequence steering vectors on spheres via Riemannian block-coordinate descent to maximize a log-determinant volume objective, improving diversity and Pass@N for best-of-N math reasoning.

Problem

Best-of-N reasoning for math is limited by output diversity collapse: sampled candidates from an LM are often near-identical, repeating the same error.

Approach

Steer hidden activations of N parallel sequences at synchronization anchors by choosing norm-constrained steering vectors that maximize the sum of squared parallelepiped volumes over all subsets, equivalent to maximizing log det(I + (H+V)^T(H+V)); solve it as Riemannian optimization over a product of spheres with block-coordinate descent and step size 1/L_i.

Key results

• On MATH500 with Qwen2.5-1.5B at temperature 0.4, SPREAD (C=10) reaches 57.6% Pass@N versus 55.8% for sampling.
• On AIME24 with Qwen2.5-Math-1.5B-Instruct at temperature 1.0, SPREAD (C=1) reaches 26.7% Pass@N versus 20.0% for sampling.
• SPREAD with C=1 yields higher unique solution counts than sampling at most temperatures, e.g. 6.97 vs 3.14 on AIME24 for Qwen2.5-1.5B at T=1.0 (sampling column shows 6.67 for that row, so gains are mixed).

Surprising findings

• Gains over sampling are often within a few points and mixed across temperatures and settings; SPREAD is not uniformly better in the tables.
• Steering is applied only at the final layer (layer 28), which the paper says is the last layer for both architectures.

Limitations

• Gains are small and inconsistent across settings, with few AIME24 problems (30) making differences noisy.
• Diversity and unique-count metrics rely on a GPT-4.1-mini judge rather than a verified measure.
• Results are limited to two 1.5B Qwen models and mathematical reasoning.

Experimental setup

Datasets: AIME24 MATH500 OlympiadBench

Benchmarks: AIME24 MATH500 OlympiadBench

Metrics: Pass@N Solution Diversity (judge score) Unique Solution Count

Baselines: temperature sampling

Builds on: activation steering Riemannian block coordinate descent

Closest papers by meaning

• Conditional Optimal Bridge for Riemannian Activation Steering (NeurIPS 2026 · similarity 0.89)
• Exploring Diverse Generation Paths via Inference-time Stiefel Activation Steering (ICLR 2026 · similarity 0.88)
• SCALE: Selective Resource Allocation for Overcoming Performance Bottlenecks in Mathematical Test-time Scaling (AAAI 2026 · similarity 0.86)
• DenseSteer: Steering Small Language Models towards Dense Math Reasoning (ICML 2026 · similarity 0.86)
• Mitigating Overthinking in Large Reasoning Models via Manifold Steering (NeurIPS 2025 · similarity 0.86)
• Reasoning on the Manifold: Bidirectional Consistency for Self-Verification in Diffusion Language Models (ICML 2026 · similarity 0.85)