Research map / Large language models / Reasoning and chain of thought

MixReasoning: Switching Modes to Think

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Long CoT dynamics and efficiency · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MixReasoning, which switches a reasoning model between detailed and concise modes within one response by scaling a LoRA concise adapter's strength, triggered by sliding-window token confidence, cutting reasoning tokens 13-49% with maintained accuracy.

Problem

Long chains of thought in large reasoning models are redundant because only a few pivotal steps are truly difficult, yet uniform reasoning depth inflates inference cost; global compression can cause underthinking on hard subproblems.

Approach

Train one concise LoRA adapter and interpolate its strength at inference time to switch between thinking and concise modes at each decoding position, with switching triggered by a hysteresis controller over sliding-window token confidence (average negative log-prob of top-k tokens), so difficult segments get detailed reasoning and routine steps are compressed.

Key results

• On Qwen3-8B, MixReasoning reaches 67.33 Pass@1 on AIME24 (vs 64.67 base) with 8873 tokens (vs 11975), a 28% reduction on average across benchmarks (5120 avg tokens vs 7027).
• Concise-mode-only with alpha=64 drops Qwen3-8B AIME24 to 30.00 Pass@1, showing uniform compression harms hard problems.
• On DeepSeek-R1-7B, MixReasoning averages 67.76 Pass@1 vs 67.33 base while cutting average tokens by 23%.

Surprising findings

• Reverse-confidence switching (thinking on high-confidence tokens) and random switching both degrade accuracy substantially, e.g. AIME24 38.33 and 43.67 vs 64.67 base on Qwen3-8B.
• Scaling the concise distillation data from 40K to 300K gave only marginal gains, suggesting long-to-short compression limits are not solved by more data.

Limitations

• Concise adapter still loses accuracy when used alone on hard benchmarks such as AIME24, so gains rely on switching back to thinking mode.
• Thresholds and window size are tuned on the evaluation benchmarks (sweeps reported on them), limiting evidence of transfer.
• Switching overhead analysis is an upper-bound argument with gamma assumed approximately 1-2 decode steps.

Experimental setup

Datasets: DeepScaleR-40K GSM8K OpenMathReasoning-300K

Benchmarks: AIME 2024 GPQA-Diamond MATH-500 GSM8K HumanEval LiveCodeBench StrategyQA CommonsenseQA

Metrics: Pass@1 average generated tokens Pass@k end-to-end latency

Baselines: BeConcise CoT-Valve NoWait DEER O1-Pruner Thinkless Concise Mode

Builds on: LoRA CoT-Valve Thinkless DeepScaleR

Closest papers by meaning

• Hint Tuning: Less Data Makes Better Reasoners (NeurIPS 2026 · similarity 0.89)
• Tracing the Traces: Latent Temporal Signals for Efficient and Accurate Reasoning (ICLR 2026 · similarity 0.89)
• The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity (NeurIPS 2025 · similarity 0.89)
• Don’t Think Longer, Think Wisely: Optimizing Thinking Dynamics for Large Reasoning Models (NeurIPS 2025 · similarity 0.89)
• Reasoning-Trace Collapse: Evaluating the Loss of Explicit Reasoning During Fine-Tuning (NeurIPS 2026 · similarity 0.88)
• LYNX: Learning Dynamic Exits for Confidence-Controlled Reasoning (NeurIPS 2026 · similarity 0.88)