Research map / Large language models / Reasoning and chain of thought
SPRINT: Enabling Interleaved Planning and Parallelized Execution in Reasoning Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Fine-tunes reasoning models on about 1,700 curated R1 math trajectories reorganized into interleaved plan and parallel-execution stages, so the model can run independent subtasks in parallel and cut sequential tokens.
Problem
Long sequential chains-of-thought in large reasoning models cause high inference latency, while existing parallel methods lack coordination or require hand-designed search structures.
Approach
Restructure natural-language reasoning traces (via step extraction, GPT-4o-mini DAG creation, and stage packing) into rounds of planning and parallel execution, fine-tune the reasoning model on them, and run an inference orchestrator where a planner emits prompts that executors process concurrently; efficiency is measured as sequential tokens.
Key results
- On MATH-500, SPRINT reaches 92.5% accuracy versus 91.0% for RFT, with 2440 vs 2880 sequential tokens.
- On problems where RFT needs over 8,000 tokens, SPRINT cuts sequential tokens by up to 39%.
- On Countdown, SPRINT reaches 85.9% accuracy with 2284 sequential tokens versus 4917 for RFT (about 53% reduction); on GPQA-Diamond it reaches 51.0% with about 10.8% fewer sequential tokens than RFT.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace sequential R1-trajectory fine-tuning with SPRINT plan–execution fine-tuning and parallel inference | DeepSeek-R1-Distill-Qwen-7B · MATH-500 | accuracy: 91.0 -> 92.5 | Helped |
| Replace sequential R1-trajectory fine-tuning with SPRINT plan–execution fine-tuning and parallel inference | DeepSeek-R1-Distill-Qwen-7B · MATH-500 | sequential tokens: 2880 -> 2440 | Helped |
| Replace sequential R1-trajectory fine-tuning with SPRINT plan–execution fine-tuning and parallel inference | DeepSeek-R1-Distill-Qwen-7B · Countdown | accuracy: 84.9 -> 85.9 | Helped |
| Replace sequential R1-trajectory fine-tuning with SPRINT plan–execution fine-tuning and parallel inference | DeepSeek-R1-Distill-Qwen-7B · Countdown | sequential tokens: 4917 -> 2284 | Helped |
| Replace sequential R1-trajectory fine-tuning with SPRINT plan–execution fine-tuning and parallel inference | DeepSeek-R1-Distill-Qwen-7B · GPQA-Diamond | accuracy: 50.5 -> 51.0 | Helped |
| Replace sequential R1-trajectory fine-tuning with SPRINT plan–execution fine-tuning and parallel inference | DeepSeek-R1-Distill-Qwen-7B · GPQA-Diamond | sequential tokens: 7103 -> 6336 | Helped |
| Use SPRINT on problems where RFT requires more than 8,000 tokens | DeepSeek-R1-Distill-Qwen-7B · MATH-500 long reasoning trajectories | sequential tokens: 39% reduction | Helped |
| Use SPRINT instead of RFT | DeepSeek-R1-Distill-Qwen-7B · MATH-500 | runtime per problem: 40.57s vs. 36.92s | Helped |
Surprising findings
- LoRA did not adequately enforce the required response format, so the authors used full fine-tuning.
- SPRINT uses about 5% more sequential tokens on short-trajectory problems due to prompt and tag overhead.
Limitations
- Wall-clock speedups were not realized; runtime is estimated from sequential tokens and TTFT, requiring optimized KV caching and multiple GPUs for parallel executors.
- Parallelism is bounded by the quality and size of the 1,700-sample SFT dataset, which is derived from math trajectories.
- Executions are treated as token sequences rather than tool calls, so tool-use parallelization is not covered.
Experimental setup
Datasets: MATH training set (R1 trajectories) MATH-500 GPQA-Diamond Countdown
Benchmarks: MATH-500 Countdown GPQA-Diamond
Metrics: pass@1 accuracy sequential tokens total tokens estimated runtime
Baselines: DeepSeek-R1-Distill-Qwen-7B RFT SoT-chat SoT-reasoning Repeated sampling with self-consistency
Builds on: DeepSeek-R1 Skeleton-of-Thought PASTA Hogwild! Inference self-consistency
Closest papers by meaning
- ThreadWeaver: Adaptive Threading for Efficient Parallel Reasoning in Language Models (ICML 2026 · similarity 0.89)
- CodePlan: Unlocking Reasoning Potential in Large Language Models by Scaling Code-form Planning (ICLR 2025 · similarity 0.89)
- Training Large Language Models To Reason In Parallel With Global Forking Tokens (ICLR 2026 · similarity 0.88)
- DTS: Enhancing Large Reasoning Models via Decoding Tree Sketching (ICML 2026 · similarity 0.88)
- InftyThink: Breaking the Length Limits of Long-Context Reasoning in Large Language Models (ICLR 2026 · similarity 0.88)
- Plan and Budget: Effective and Efficient Test-Time Scaling on Reasoning Large Language Models (ICLR 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.