Research map / Large language models / Reasoning and chain of thought

DREAM-R: Multimodal Speculative Reasoning with RL-Based Refined Drafting, Precise Verification, and Fully Parallel Execution

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Speculative decoding · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DREAM-R, a multimodal speculative reasoning framework with RL-trained draft models (SAPO), a ratio-based step verifier (CPN), and fully parallel draft/target/verify execution (FPSR), reporting up to 2.48x speedup with near-target accuracy.

Problem

Speculative reasoning for multimodal large reasoning models suffers from misalignment between draft reasoning steps and target-verified reasoning, and existing methods lose accuracy and acceptance when the draft is a weaker vision-language model.

Approach

Trains the draft model with GRPO-style RL using rewards for target-verified outcome, draft acceptance rate, and length; verifies steps with a probability-ratio criterion s+/(s+ + s-) > 0.7 from the target model's positive/negative keywords; and overlaps drafting, target generation, and verification with rollback on rejection.</NEW>

Key results

• DREAM-R with Qwen3-VL-32B target and Qwen3-VL-2B draft reaches up to 2.48x speedup on MMBench and RealWorldQA.
• DREAM-R with Qwen3-VL-32B target and Qwen3-VL-2B draft achieves 75.98% on MathVerse (vs 76.00% for the target) at 2.38x speedup.
• SpecReason's accuracy drops from 78% to 43.2% in the diagnostic study on MathVista with Qwen3-VL-32B target.

Surprising findings

• Directly applying SpecReason's score-based verification to a VLM target drops accuracy sharply (78% to 43.2% in diagnostic), and for the MiMo-VL-7B-RL draft with Qwen3-VL-235B target SpecReason reaches only 0.71x on MMBench.

Limitations

• Speedups depend on the draft-target pairing and acceptance rates vary widely across benchmarks (e.g., 31% to 68%).
• Target model is accessed via API during training rollouts, so training depends on external model access.
• Evaluation is limited to four benchmarks and one target family (Qwen3-VL) plus a few drafts.

Experimental setup

Datasets: Geo3K OCR-VQA ScienceQA

Benchmarks: MathVerse MMBench RealWorldQA MMMU

Metrics: accuracy acceptance rate speedup

Baselines: Standard SD SpecReason LR DREAM-R-NS

Builds on: SpecReason Lookahead Reasoning GRPO speculative decoding

Closest papers by meaning

• Steering Pretrained Drafters During Speculative Decoding (AAAI 2026 · similarity 0.87)
• Learning To Draft: Adaptive Speculative Decoding with Reinforcement Learning (ICLR 2026 · similarity 0.87)
• D-PACE: Dynamic Position-Aware Cross-Entropy for Parallel Speculative Drafting (NeurIPS 2026 · similarity 0.87)
• EfficientRollout: System-Aware Self-Speculative Decoding for RL Rollouts (NeurIPS 2026 · similarity 0.87)
• SpecReason: Fast and Accurate Inference-Time Compute via Speculative Reasoning (NeurIPS 2025 · similarity 0.86)
• SpecExit: Accelerating Large Reasoning Model via Speculative Exit (ICML 2026 · similarity 0.86)