Research map / Large language models / Reasoning and chain of thought
R2R: Efficiently Navigating Divergent Reasoning Paths with Small-Large Model Token Routing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes R2R, a token-level router that sends only path-divergent tokens from a small reasoning model (R1-1.5B) to a large one (R1-32B), using labels from a sentence-level path-following pipeline with an LLM verifier.
Problem
Distilled small reasoning models diverge from LLM reasoning paths and lose accuracy, while running the full LLM is expensive at test time; query-level routing and speculative decoding are coarse or inefficient for this.
Approach
Label each SLM-LLM next-token difference as neutral or divergent by continuing both candidates with the LLM to sentence end and asking an LLM verifier whether the reasoning meaningfully differs; train a 56M-parameter FFN router on SLM logits, token embedding and hidden states to invoke the LLM only for divergent tokens, with immediate correction and no rollback.
Key results
- R2R averages 46% accuracy at 5.6B average activated parameters across AIME, LiveCodeBench and GPQA, versus 43% for R1-14B at 14B and 28% for R1-7B at 7B.
- R2R achieves 2.76x generation speedup over R1-32B and 1.62x over R1-14B on AIME with two A800 GPUs.
- Routing uses the LLM for only about 11-15% of tokens while raising R1-1.5B accuracy 4.6x (10% to 46% average).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| route only divergent tokens using R2R rather than use R1-14B | R1-1.5B + R1-32B · Average across AIME, LiveCodeBench, and GPQA | accuracy: 43% -> 46% | Helped |
| route only divergent tokens using R2R rather than use R1-1.5B | R1-1.5B + R1-32B · Average across AIME, LiveCodeBench, and GPQA | accuracy: 10% -> 46% | Helped |
| use R2R rather than R1-32B | AIME inference · AIME | wall-clock speed: 2.76× | Helped |
| route all different tokens instead of only divergent tokens (ablation) | R2R · AIME | accuracy: 55% -> 40% | Hurt |
| add SLM logits to router inputs (ablation) | R2R · AIME | accuracy: 47% -> 55% | Helped |
| add token embeddings to router inputs (ablation) | R2R · AIME | accuracy: 42% -> 55% | Helped |
| use hidden states and token embeddings without logits rather than all router inputs (ablation) | R2R · AIME | accuracy: 55% -> 47% | Hurt |
| use hidden states only rather than all router inputs (ablation) | R2R · AIME | accuracy: 55% -> 42% | Hurt |
| use sentence-level path-following with Qwen2.5-72B verifier rather than R1-7B | Path-following routing · AIME 2024 within 8K tokens | correct answers: 8 -> 16 | Helped |
Surprising findings
- The 1.5B SLM differs from the 32B LLM on only about 11% of tokens, far fewer than the divergence seen in final answers, and only about 4.9% of tokens are truly divergent.
- Labeling with a neutral-tolerant objective matters: training the router on all different tokens drops accuracy from 55% to 40% at similar LLM usage.
Limitations
- Routing strategy is tuned and tested mainly for greedy decoding; sampling extension is only partially validated.
- Relies on an LLM verifier (Qwen2.5-72B) whose quality affects labels; smaller verifiers reduce recall.
- System-level optimization still needed to realize full cost benefits; occasional output inconsistencies versus LLM for very long outputs.
Experimental setup
Datasets: AIME Bespoke-Stratos-17k GPQA-Extended
Benchmarks: AIME 2024-2025 LiveCodeBench GPQA-Diamond Arena-Hard MMLU-Redux-Philosophy
Metrics: accuracy average activated parameters cost wall-clock latency memory access TFLOPs
Baselines: R1-1.5B R1-7B R1-14B R1-32B QR-SW QR-LLM QR-BERT QR-MF EAGLE2 HASS
Builds on: RouteLLM speculative decoding EAGLE2 HASS DeepSeek-R1-Distill-Qwen
Closest papers by meaning
- Confidence-Guided Stepwise Model Routing for Cost-Efficient Reasoning (AAAI 2026 · similarity 0.91)
- Exploring the Limits of Sub-Billion Language Model Reasoners with Open Training Recipes (ICLR 2026 · similarity 0.90)
- Think Deep, Not Just Long: Measuring LLM Reasoning Effort via Deep-Thinking Tokens (ICML 2026 · similarity 0.90)
- Token Assorted: Mixing Latent and Text Tokens for Improved Language Model Reasoning (ICML 2025 · similarity 0.89)
- Dr.LLM: Dynamic Layer Routing in LLMs (ICLR 2026 · similarity 0.89)
- THE PATH OF LEAST RESISTANCE: GUIDING LLM REASONING TRAJECTORIES WITH PREFIX CONSENSUS (ICLR 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.