AI Paper Atlas

Research map / Architectures and efficiency / RNNs and normalization

WarmServe: Enabling One-for-Many GPU Prewarming for Multi-LLM Serving

Chiheng Lou, Sheng Qi, Rui Kang, Yong Zhang, Chen Sun, pengcheng wang, Xuanzhe Liu, Xin Jin

ICML 2026 · Architectures and efficiency › RNNs and normalization · approach: Distributed LLM inference systems · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add one-for-many GPU prewarming (weights in GPU memory) versus on-demand engine creationWarmServe · Llama2-7B/13B/70B cold-startTTFT: 60.13x-87.63x TTFT reductionImproved
use WarmServe instead of SLLM-GPU autoscalingmulti-LLM serving · AzureConv-based workload, RPS 10-25P99 TTFT: 1.53x-50.79x reductionImproved
add proactive prewarming into unused KV cache of GPUs about to be released (ablation)WarmServe · AzureConv-based workloadtail TTFT: 1.03x-32.87x reductionImproved
use WarmServe instead of MuxServe GPU sharing under heavy loadmulti-LLM serving · RPS 15, alpha=0.5P95/P99 TTFT: MuxServe 34.5x and 134.0x higher P95 and P99 TTFTImproved
add model prewarming (ablation)WarmServe · RPS 25, 100ms TTFT thresholdfraction of requests within 100ms: without it 0.15x of baselineImproved
add model placement algorithm instead of round robin (ablation)WarmServe · RPS 25, 100ms TTFT thresholdfraction of requests within 100ms: without it 0.29x of baselineImproved
use 5-minute prediction window instead of 3 or 40 minutes (ablation)WarmServe · RPS 25, 100ms TTFT thresholdfraction of requests within 100ms: 3 min 0.46x, 40 min 0.30x of 5-min baselineImproved
add one-for-many prewarming versus predictive autoscaling without prewarming at high load512-GPU simulation · RPS 800P99 TTFT (s): 4.42 -> 0.38Improved
use predictive autoscaling instead of prewarming at light load512-GPU simulation · RPS 320 and 480P99 TTFT (s): 0.18 vs 0.18 and 0.25 vs 0.25No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy