Research map / Large language models / Reasoning and chain of thought
Lookahead Routing for Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Lookahead, a response-aware LLM router that predicts latent representations of each candidate model's response (via a causal or masked LM) without decoding, and jointly trains routing classification with response reconstruction.
Problem
Existing LLM routers classify queries using only the input, ignoring how candidate models would actually respond, which leads to suboptimal routing for complex or ambiguous queries; generating full responses for the router is infeasible at inference time.
Approach
Train the router to 'foresee' candidate responses by predicting their latent representations from model-ID tokens (sequence-level CLM with MID hidden states, token-level MLM with masked response spans and curriculum end-masking), jointly optimizing BCE routing loss with a response reconstruction loss, so no full decoding is needed at inference.
Key results
- MLM-based Lookahead achieves average normalized score 40.8 across seven benchmarks, versus 37.9 for the best baseline RouterDC (MLM), a 7.7% gain as claimed.
- CLM-based Lookahead reaches 37.0 average normalized score, exceeding SMOOTHIE (35.4) by 4.5% in average normalized score.
- Removing response modeling drops performance by 6.2 (CLM) and 6.8 (MLM) points; with ~16% of training data the CLM w/ RM matches the full w/o RM baseline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Lookahead instead of RouterDC | ModernBERT-base router · Seven-benchmark average | normalized score: 37.9 to 40.8 | Helped |
| Use Lookahead instead of MLC | SmolLM2-135M router · Seven-benchmark average | normalized score: 26.1 to 37.0 | Helped |
| Add response modeling (ablation) | Lookahead with CLM · Seven-benchmark average | normalized score: +6.2 points | Helped |
| Add response modeling (ablation) | Lookahead with MLM · Seven-benchmark average | normalized score: +6.8 points | Helped |
| Use curriculum masking (ablation) | MLM-based Lookahead · Seven-benchmark average | normalized score: +4.7 points | Helped |
| Use end-masking instead of start-masking or random masking (ablation) | MLM-based Lookahead · Seven-benchmark average | normalized score | Helped |
| Predict all candidate responses jointly instead of per-model separately (ablation) | MLM-based Lookahead · Instruction-following benchmarks | routing performance | Helped |
| Increase response-modeling weight lambda within the tested range (ablation) | Lookahead CLM and MLM · Seven-benchmark average | normalized score | No clear effect |
| Increase repeated model ID tokens up to m=64 (ablation) | MLM-based Lookahead · Seven-benchmark average | normalized score | Helped |
| Increase repeated model ID tokens beyond m=64 (ablation) | MLM-based Lookahead · Seven-benchmark average | normalized score | Hurt |
| Set curriculum fraction alpha above 0.4 (ablation) | MLM-based Lookahead · Seven-benchmark average | normalized score | Hurt |
| Add response modeling (ablation) | CLM-based Lookahead · Training efficiency | data efficiency: 6.3× gain in data efficiency | Helped |
Surprising findings
- The CLM variant is comparable to the MLM variant on math and code but much weaker on open-ended instruction following, which the authors attribute to isolated per-model likelihood scoring.
- Adding more candidate models (from 5 to 8) slightly reduced performance, attributed to weaker or redundant models adding routing noise.
Limitations
- Does not explicitly account for cost trade-offs between large and small models.
- Models only the first m tokens of responses, which makes math responses that restate the problem look similar and hurts routing.
- Router may learn to favor biased or factually incorrect outputs if the reward model fails to detect them.
Experimental setup
Datasets: UltraFeedback OpenMathInstruct-2 Self-OSS-Instruct-SC2
Benchmarks: AlpacaEval-2 Arena-Hard MT-Bench GSM8K MATH HumanEval MBPP
Metrics: original score normalized score (gap closed between random and oracle routi
Baselines: kNN k-means SMOOTHIE MLC ZOOTER RouterDC Reward Model Select
Builds on: RouterDC ZOOTER SMOOTHIE ModernBERT SmolLM2
Closest papers by meaning
- Confidence-Guided Stepwise Model Routing for Cost-Efficient Reasoning (AAAI 2026 · similarity 0.91)
- Universal Model Routing for Efficient LLM Inference (ICLR 2026 · similarity 0.91)
- RouteLLM: Learning to Route LLMs from Preference Data (ICLR 2025 · similarity 0.90)
- BEST-Route: Adaptive LLM Routing with Test-Time Optimal Compute (ICML 2025 · similarity 0.89)
- Routing Experts: Learning to Route Dynamic Experts in Existing Multi-modal Large Language Models (ICLR 2025 · similarity 0.89)
- GraphRouter: A Graph-based Router for LLM Selections (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.