Research map / Large language models / Reasoning and chain of thought

Short Chains, Deep Thoughts: Balancing Reasoning Efficiency and Intra-Segment Capability via Split-Merge Optimization

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Efficient and adaptive thinking · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes CoSMo, which curates segment-structured reasoning data via LLM-driven split-merge refinement, then applies SFT and GRPO with a segment-count penalty (not token-length) to shorten multi-hop QA reasoning chains while improving accuracy.

Problem

Long reasoning chains in LRMs incur high latency and cost, and existing efficiency methods that penalize token length or prune steps can disrupt logical coherence and hurt accuracy on knowledge-intensive multi-hop QA.

Approach

Treat reasoning as discrete segments and optimize segment count toward the gold hop count rather than total tokens: a split-merge algorithm (LLM judge + generator) curates logic-dense SFT data, and GRPO uses a segment-deviation penalty with a one-step tolerance while leaving intra-segment length unconstrained.

Key results

• CoSMo reaches 71.7% average accuracy vs 68.9% for CoT on Llama-3.1-8B-Instruct, with average tokens 153 vs 217 and segments 2.9 vs 5.4.
• CoSMo reduces average segments by 28.7% relative to efficient reasoning baselines, and reports a 3.3-point accuracy gain.
• On MuSiQue, CoSMo's accuracy drops only 2.1 points from 2-hop to 4-hop, versus 3.9 for CoT and 4.0 for C3oT.

Surprising findings

• Accuracy peaks when segment count matches the ground-truth hop count, and declines when extra segments are added, so efficiency is not monotonic in brevity.
• Length-penalty RL (LCPO) reduced tokens but increased segment count and gave only marginal accuracy gains.

Limitations

• Relies on gold hop count k* for the reward and curation, which requires annotated reasoning paths.
• Split-merge curation uses a 72B LLM judge and generator, which is costly; evaluation uses LLM judges rather than human evaluation.
• Evaluated mainly on multi-hop QA; generalization to math or code reasoning is not shown.

Experimental setup

Datasets: HotpotQA HaluEval

Benchmarks: HotpotQA HaluEval NQ CRAG MuSiQue

Metrics: accuracy average token count average segment count

Baselines: CoT ToT HTP CoD TALE C3oT FS-BoN SPIRIT LCPO ThinkPrune

Builds on: GRPO C3oT ThinkPrune LCPO Chain-of-Thought

Closest papers by meaning

• OS-Pruner: Pruning Chains-of-Thought of Reasoning Models via Optimal Stopping (NeurIPS 2026 · similarity 0.90)
• SLAT: Segment-Level Adaptive Trimming for Efficient CoT Reasoning (ICML 2026 · similarity 0.89)
• Towards Efficient Large Language Reasoning Models via Extreme-Ratio Chain-of-Thought Compression (ICML 2026 · similarity 0.89)
• DeepCompress: A Dual Reward Strategy for Dynamically Exploring and Compressing Reasoning Chains (ICLR 2026 · similarity 0.89)
• TERMINATOR: Learning Optimal Exit Points for Early Stopping in Chain-of-Thought Reasoning (NeurIPS 2026 · similarity 0.89)
• Rectifying LLM Thought from Lens of Optimization (ICLR 2026 · similarity 0.89)