Research map / Large language models / Reasoning and chain of thought

Native Parallel Reasoner: Reasoning in Parallelism via Self-Distilled Reinforcement Learning

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Efficient LLM serving · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Native Parallel Reasoner (NPR), a teacher-free three-stage pipeline (self-distilled format RL, parallel SFT with parallel attention mask/positional encoding, and PAPO parallel RL) on Qwen3-4B that yields genuinely parallel reasoning with up to 4.6x speedup.

Problem

Existing parallel reasoning methods rely on supervised distillation from stronger teachers, often fall back to autoregressive decoding, and their inference engines do not enforce strict parallel semantics or stable RL rollouts.

Approach

Self-evolve parallel reasoning without external teachers: DAPO with a format-aware reward induces the Map-Process-Reduce format, rejection-sampled self-distillation feeds parallel SFT with a parallel attention mask and position IDs, and PAPO (on-policy, batch-level advantage, no clip-masking on special tokens, schema-level filtering) performs RL within the parallel execution graph on a fixed NPR E

Key results

• NPR on Qwen3-4B (non-thinking) reaches avg 64.4 across eight benchmarks vs 39.9 for the base model, with AIME25 rising from 19.1 to 53.8.
• NPR achieves up to 4.6x speedup over sequential decoding on AIME25 (2979.8 vs 646.8 TPS), versus 2.4x for Multiverse.
• NPR shows a 100% parallel trigger rate on all eight benchmarks, compared with 45.8%-70% for MV-32B.

Surprising findings

• Naively enforcing the parallel format at the outset drops Qwen3-4B-Instruct-2507 on AIME25 from 47.5 to 17.5.
• Multiverse-style baselines fall back to autoregressive decoding in 30%+ of test cases.

Limitations

• Gains demonstrated on a single 4B backbone with an 8k training subset.
• Speedups are task-dependent (2.9x on AMC23 vs 4.6x on AIME25).
• Baseline comparisons partly use reproduced Multiverse-4B and original reported numbers, so not fully controlled.

Experimental setup

Datasets: ORZ s1.1-8k

Benchmarks: AIME25 AIME24 HMMT25 OlympiadBench Minerva-Math ZebraLogic AMC23 MATH500

Metrics: avg@8 avg@1 best@8 tokens per second speedup parallel trigger rate

Baselines: Multiverse-32B Multiverse-4B Qwen3-4B-Instruct-2507 Qwen3-4B (non-thinking) Qwen2.5-32B-Instruct SR (sequential RL) SR-Beta

Builds on: DAPO Multiverse GRPO Lite-PPO SGLang

Closest papers by meaning

• ThreadWeaver: Adaptive Threading for Efficient Parallel Reasoning in Language Models (ICML 2026 · similarity 0.91)
• Parallel-R1: Towards Parallel Thinking via Reinforcement Learning (ICLR 2026 · similarity 0.89)
• Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models (ICML 2026 · similarity 0.88)
• Native Reasoning Models: Training Language Models to Reason on Unverifiable Data (ICLR 2026 · similarity 0.88)
• A$^2$FM: An Adaptive Agent Foundation Model for Tool-Aware Hybrid Reasoning (ICLR 2026 · similarity 0.88)
• AREAL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoning (NeurIPS 2025 · similarity 0.87)