Research map / Bandits and online learning / Online convex optimization and games
Near-Optimal Online Deployment and Routing for Streaming LLMs
Shaoang Li, Jian Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| optimistic (UCB/LCB) stage-wise deployment plus routing LP instead of greedy or uniform baselines | StageRoute · RouterBench (36,497 queries, 11 LLMs) | cumulative regret | Improved |
| uniform random deployment instead of StageRoute | uniform sampling baseline · RouterBench | budget adherence | Hurt |
| greedy top-M_max by UCB/LCB ratio instead of LP-based optimistic deployment | greedy baseline · RouterBench | performance-cost operating point | Hurt |
| vary deployment update interval (500, 1000, 2000 rounds) (ablation) | StageRoute · RouterBench, M_max=5, b=0.001 | cumulative regret | Mixed |
| vary concurrency cap M_max (3, 5, 10) (ablation) | StageRoute · RouterBench, b=0.001 | cumulative regret | No clear effect |
| relax budget (b=0.002 vs 0.001) (ablation) | StageRoute · RouterBench | cumulative regret | Hurt |
Closest papers by meaning
- BEST-Route: Adaptive LLM Routing with Test-Time Optimal Compute (ICML 2025 · similarity 0.89)
- MESS+: Dynamically Learned Inference-Time LLM Routing in Model Zoos with Service Level Guarantees (NeurIPS 2025 · similarity 0.88)
- A Unified Approach to Routing and Cascading for LLMs (ICML 2025 · similarity 0.87)
- RACER: Risk-Aware Calibrated Efficient Routing for Large Language Models (ICML 2026 · similarity 0.87)
- Efficient Training-Free Online Routing for High-Volume Multi-LLM Serving (NeurIPS 2025 · similarity 0.87)
- TokenRouter: Efficient Serving System for Token-Level LLM Routing (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
