Research map / Large language models / Reasoning and chain of thought
A Unified Approach to Routing and Cascading for LLMs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Derives optimal routing and cascading strategies as linear optimization problems, proves optimality of a new cascade strategy, and proposes cascade routing, a unified framework that interleaves routing and cascading, evaluated on RouterBench and SWE-Bench and other benchmarks.
Problem
Existing LLM model selection strategies (routing and cascading) lack optimality proofs, do not reveal when they are effective, and cannot combine both paradigms to improve the cost-performance tradeoff.
Approach
Frame routing and cascading as linear optimization over probabilistic strategies; cascading becomes sequential routing over supermodels (model prefixes), and cascade routing lets the router pick any supermodel at each step, yielding a provably optimal unified strategy whose performance hinges on ex-ante and post-hoc quality estimators.
Key results
- On SWE-Bench, cascade routing outperforms baseline strategies by up to 14% AUC.
- On RouterBench, cascade routing improves by up to 8% over cascading and up to 12% over routing under varying noise levels.
- On classification and open-form tasks with poor quality estimates, cascade routing performs on par or better than baselines with gains up to 1.2%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use cascade routing instead of routing or cascading | Model selection on RouterBench · RouterBench | AUC: performance gains between $1\%$ to $4\% | Helped |
| Use cascade routing instead of the baseline cascade | Model selection on RouterBench · RouterBench, five models across noise levels | AUC: up to an $8\%$ improvement for cascading | Helped |
| Use cascade routing instead of routing | Model selection on RouterBench · RouterBench, five models across noise levels | AUC: up to a $12\%$ improvement for routing | Helped |
| Use cascade routing instead of baseline strategies | Model selection on SWE-Bench · SWE-Bench | AUC: up to $14\% | Helped |
| Use routing instead of the trivial baseline | Model selection with insufficient ex-ante quality estimates · SWE-Bench | AUC | Hurt |
| Use the baseline threshold cascade instead of routing | Model selection with binary post-hoc feedback · SWE-Bench | AUC | Hurt |
| Use routing instead of the baseline cascade | Qwen model selection with accurate ex-ante quality estimates · Minerva Math and LiveCodeBench | AUC: $10\%$ | Helped |
| Use cascade routing instead of other model-selection baselines | Llama and Gemma model selection · Classification and Open-Form | AUC: up to $1.2\%$ | Helped |
Surprising findings
- Routing does not outperform the trivial linear-interpolation baseline on SWE-Bench, since ex-ante quality estimates are insufficient.
- The threshold-based baseline cascade performs poorly on SWE-Bench even with perfect post-hoc estimates, due to binary feedback.
Limitations
- Performance depends heavily on quality estimator accuracy; poor estimates give narrow gains.
- Cascade routing requires evaluating up to 2^k supermodels, though pruning mitigates this.
- Main experiments rely on simulated noise on RouterBench rather than real estimators.
Experimental setup
Datasets: RouterBench SWE-Bench Minerva Math LiveCodeBench ARC-Challenge MMLU-Pro MixEval GSM8k
Benchmarks: RouterBench SWE-Bench Minerva Math LiveCodeBench ARC-Challenge MMLU-Pro MixEval GSM8k
Metrics: AUC of quality-cost curve
Baselines: Linear Interpolation Routing Cascade (threshold-based baseline)
Builds on: RouterBench threshold-based cascading optimal routing (Hu et al. style deterministic routing)
Closest papers by meaning
- Cascadia: An Efficient Cascade Serving System for Large Language Models (ICLR 2026 · similarity 0.91)
- BEST-Route: Adaptive LLM Routing with Test-Time Optimal Compute (ICML 2025 · similarity 0.90)
- Lookahead Routing for Large Language Models (NeurIPS 2025 · similarity 0.89)
- RACER: Risk-Aware Calibrated Efficient Routing for Large Language Models (ICML 2026 · similarity 0.88)
- DiSRouter: Distributed Self-Routing for LLM Selections (ICLR 2026 · similarity 0.88)
- Confidence-Guided Stepwise Model Routing for Cost-Efficient Reasoning (AAAI 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.