Research map / Large language models / Reasoning and chain of thought

Training Large Language Models To Reason In Parallel With Global Forking Tokens

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Efficient and adaptive thinking · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Set Supervised Fine-Tuning (SSFT), which uses Hungarian bipartite matching between reserved global forking tokens and diverse reasoning traces to learn a set NTP loss, plus a short GFPO RL step on the forking tokens, improving pass@1 and consensus on math and code reasoning.

Problem

Parallel test-time scaling needs diverse yet correct reasoning paths, but temperature-based diversity trades off against accuracy, and SFT on multiple traces collapses distinct reasoning modes.

Approach

Treat parallel reasoning as a set-of-next-token-prediction problem: reserved global forking tokens are matched to distinct reasoning traces via minimum-cost bipartite matching (Hungarian algorithm) on NTP losses, and only matched pairs are trained, so each forking token learns a distinct reasoning mode; GFPO then runs RL only on forking-token policy gradients.

Key results

• SSFT-32B reaches 64.06 pass@1 on AIME24 and 58.13 on AIME25, versus 58.23 and 51.96 for SFT-mixed-distill-32B (multi-trace SFT baseline).
• SSFT-32B with cons@6 reaches 73.94 on AIME25 and cons@32 reaches 86.67 on AIME25.
• SSFT-32B-GFPO reaches 42.10 pass@1 on LiveCodeBench v5 versus 32.34 for SFT-mixed-distill-32B.

Surprising findings

• Randomly assigning global forking tokens during SFT yields no meaningful difference in reasoning length or accuracy across tokens, while optimal matching produces distinct modes.
• GRPO on full sequences does not improve and can slightly degrade performance in the small-scale RL setting, while GFPO on forking tokens does.

Limitations

• Requires multiple reasoning traces per question from diverse teachers, which may be costly to obtain.
• Gains are smaller on Llama3.1-8B and Qwen3-4B than on Qwen2.5-32B.
• Evaluated mainly at small scale (1k training questions); RL step uses only 10 steps on 1,280 questions.

Experimental setup

Datasets: s1k DAPO-Math-17k Open-Thoughts Intellect-2-RL OpenR1-93k

Benchmarks: AIME 2024 AIME 2025 MATH-500 GPQA-Diamond LiveCodeBench v5

Metrics: pass@1 cons@6 cons@32 pass@k

Baselines: s1.1-32B Multiverse-32B SFT-OSS-distill-32B SFT-mixed-distill-32B-tags SSFT-32B (random σ) GRPO

Builds on: DETR GRPO s1k Hungarian algorithm

Closest papers by meaning

• ThreadWeaver: Adaptive Threading for Efficient Parallel Reasoning in Language Models (ICML 2026 · similarity 0.89)
• SPRINT: Enabling Interleaved Planning and Parallelized Execution in Reasoning Models (NeurIPS 2025 · similarity 0.88)
• The First Few Tokens Are All You Need: An Efficient and Effective Unsupervised Prefix Fine-Tuning Method for Reasoning Models (NeurIPS 2025 · similarity 0.88)
• Native Parallel Reasoner: Reasoning in Parallelism via Self-Distilled Reinforcement Learning (ICML 2026 · similarity 0.87)
• Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach (NeurIPS 2025 · similarity 0.87)
• Rethinking Fine-Tuning when Scaling Test-Time Compute: Limiting Confidence Improves Mathematical Reasoning (NeurIPS 2025 · similarity 0.87)