Research map / Large language models / Reasoning and chain of thought

SwiReasoning: Switch-Thinking in Latent and Explicit for Pareto-Superior Reasoning LLMs

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Search and self-correction in reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SwiReasoning, a training-free decoding framework that switches between explicit CoT and soft-embedding latent thinking based on entropy trends and caps the number of latent-to-explicit switches to curb overthinking.

Problem

Training-free latent reasoning (soft thinking) diffuses probability mass across many implicit paths, hurting accuracy, and still overthinks, wasting tokens; explicit CoT collapses distributions to single tokens.

Approach

Dynamically alternate between latent (soft-embedding) and explicit thinking blocks using a reference-entropy trend as a block-wise confidence signal (switch to explicit when entropy drops below the block reference, to latent after a dwell window when it rises), plus a switch-count controller that forces convergence and early answering after a budget of latent-to-explicit switches.

Key results

• On Qwen3-8B, SwiR improves average Pass@1 over CoT by 2.03 points across math/STEM benchmarks (80.94 vs 78.91), with +5.00 on AIME 2025 for Qwen3-1.7B.
• Across models, average accuracy gains over CoT are reported as 1.8-3.1 points; on Qwen3-32B the gain is +1.92 on average.
• Under constrained budgets, token efficiency improves by 57%-79% on average, with peak efficiency gains of 4.6x-6.8x over CoT.

Surprising findings

• Greedy CoT often underperforms sampled CoT, and Soft Thinking underperforms CoT on several models, showing pure latent reasoning hurts accuracy without switching.
• The method struggles on spatial tasks with rigid geometric constraints, as latent smoothing blurs constraints.

Limitations

• Struggles with tasks requiring precise constraint satisfaction, such as geometric topology problems.
• Hyperparameters (alpha_0, beta_0, window size) are tuned on benchmarks and alpha_0 is task-dependent, exposed for user adjustment.
• Evaluated only on Qwen3 and DeepSeek-R1-Distill-Llama models with a single attempt per configuration for most benchmarks.

Experimental setup

Datasets: GSM8K MATH500 AIME 2024 AIME 2025 GPQA Diamond HumanEval LeetCode-Contest MBPP LiveCodeBench 2WikiMultihopQA CommonsenseQA

Benchmarks: GSM8K MATH500 GPQA Diamond AIME 2024 AIME 2025 HumanEval LeetCode-Contest MBPP LiveCodeBench 2WikiMultihopQA CommonsenseQA

Metrics: Pass@1 token efficiency Pass@k latency TFLOPs

Baselines: CoT (sampling) CoT (greedy) Soft Thinking

Builds on: Soft Thinking chain-of-thought Qwen3 HuggingFace transformers

Closest papers by meaning

• Token Assorted: Mixing Latent and Text Tokens for Improved Language Model Reasoning (ICML 2025 · similarity 0.91)
• SABER: Switchable and Balanced Training for Efficient LLM Reasoning (AAAI 2026 · similarity 0.90)
• Sample Smart, Not Hard: Correctness-First Decoding for Better Reasoning in LLMs (ICLR 2026 · similarity 0.89)
• Reasoning via Test-Time Instance-Level Policy Gradient in Latent Space (NeurIPS 2026 · similarity 0.89)
• A Theoretical Study on Bridging Internal Probability and Self-Consistency for LLM Reasoning (NeurIPS 2025 · similarity 0.89)
• Thinking on the Fly: Test-Time Reasoning Enhancement via Latent Thought Policy Optimization (ICLR 2026 · similarity 0.89)