Research map / Large language models / Reasoning and chain of thought

R2-Router: A New Paradigm for LLM Routing with Reasoning

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Routing, pruning and distillation for reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes R2-Router, which jointly selects an LLM and an output-length budget (enforced via "use at most k tokens" instructions) using per-budget quality predictors, plus the R2-Bench routing dataset with multi-budget responses from 15 LLMs.

Problem

Existing LLM routers treat each LLM as a single fixed quality-cost point per query, ignoring that quality varies with output length, so they exclude powerful LLMs whose estimated cost exceeds budget.

Approach

Treat output-length budget as a controllable variable: predict a quality-cost curve per LLM (multi-head predictor over anchor budgets) and search over (LLM, budget) pairs, enforcing the budget via length-constrained prompts; also release a dataset recording responses across budgets.

Key results

• R2-Router reaches average quality 0.8 at about 0.5e-3 cost, where reactive baselines need 4-5x that budget.
• On R2-Bench, Oracle AUDC rises from 0.85 (SPROUT) to 0.98 and Peak Quality from 0.90 to 0.98.
• Integrating with UniRouter (Uni-R2Router) raises AUDC from 0.590 to 0.623 and reduces QNC by 80% on a dynamic LLM pool.

Surprising findings

• Small models (below 4B) show only 3%-21% compliance with the 10-token length constraint, yet the method still gains by leveraging large models under constraints.
• Prompting reactive baselines with a conciseness instruction does not close the gap, since their selection logic still excludes large models.

Limitations

• Small models often fail to follow length constraints at tight budgets, so the approach mainly helps large models.
• Quality labels come from a single LLM judge, with only partial robustness checks against another judge.
• Dataset construction requires collecting responses from many LLMs under many budgets, which is costly.

Experimental setup

Datasets: MMLU-Pro OpenHermes MATH GPQA MuSR RAGBench SPROUT RouterArena

Benchmarks: R2-Bench RouterArena MMLU-Pro (STEM/non-STEM OOD split)

Metrics: AUDC QNC Peak Quality Arena Score

Baselines: MIRT-IRT NIRT-IRT CARROT-KNN CARROT-Linear UniRouter SPROUT FrugalGPT AutoMix

Builds on: SPROUT UniRouter RouterArena CARROT

Closest papers by meaning

• Router-R1: Teaching LLMs Multi-Round Routing and Aggregation via Reinforcement Learning (NeurIPS 2025 · similarity 0.90)
• Dr.LLM: Dynamic Layer Routing in LLMs (ICLR 2026 · similarity 0.90)
• BEST-Route: Adaptive LLM Routing with Test-Time Optimal Compute (ICML 2025 · similarity 0.89)
• TokenRouter: Efficient Serving System for Token-Level LLM Routing (NeurIPS 2026 · similarity 0.89)
• Models Under SCOPE: Scalable and Controllable Routing via Pre-hoc Reasoning (ICML 2026 · similarity 0.88)
• Causal LLM Routing: End-to-End Regret Minimization from Observational Data (NeurIPS 2025 · similarity 0.87)