Research map / Vision-language and multimodal / Multimodal LLMs
ElasticMM: Efficient Multimodal LLMs Serving with Elastic Multimodal Parallelism
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Elastic Multimodal Parallelism and the ElasticMM serving system, which separates text-only and multimodal requests into modality groups and elastically scales encode/prefill/decode stages, reducing TTFT up to 4.2x and raising throughput 3.2-4.5x over vLLM.
Problem
Serving multimodal LLMs is inefficient because tightly coupled architectures mix text-only and multimodal requests and use fixed parallelism across inference stages, which raises TTFT and lowers resource utilization under bursty multimodal workloads.
Approach
Two-level decoupling: modality-level groups with burst-tolerance-based proactive and reactive load balancing, plus stage-level elastic partition scheduling that varies parallelism per stage (encode, prefill, decode), combined with unified multimodal prefix caching and non-blocking encoding.
Key results
- On ShareGPT-4o, ElasticMM reduces TTFT by up to 4.2x for Qwen2.5-VL and 3.5x for LLaMA3.2-Vision versus baselines.
- ElasticMM achieves 3.2x (LLaMA3.2-Vision) to 4.5x (Qwen2.5-VL) higher throughput than vLLM on ShareGPT-4o under SLO constraints.
- ElasticMM achieves up to 2.3x higher throughput than DistServe.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ElasticMM instead of vLLM | Qwen2.5-VL-7B serving · ShareGPT-4o | TTFT: 4.2× | Helped |
| Use ElasticMM instead of vLLM | LLaMA3.2-Vision-11B serving · ShareGPT-4o | TTFT: 3.5× | Helped |
| Use ElasticMM instead of vLLM | Qwen2.5-VL-7B serving · VisualWebInstruct | TTFT: 3.7× | Helped |
| Use ElasticMM instead of vLLM | LLaMA3.2-Vision-11B serving · VisualWebInstruct | TTFT: 2.9× | Helped |
| Use ElasticMM instead of vLLM | Qwen2.5-VL-7B serving · ShareGPT-4o | throughput: 4.5× | Helped |
| Use ElasticMM instead of vLLM | LLaMA3.2-Vision-11B serving · ShareGPT-4o | throughput: 3.2× | Helped |
| Enable EMP rather than static resource allocation (ablation) | Qwen2.5-VL-7B serving · ShareGPT-4o | P90 effective throughput: 1.8× | Helped |
| Enable EMP rather than static resource allocation (ablation) | LLaMA3.2-Vision-11B serving · ShareGPT-4o | P90 effective throughput: 2.3× | Helped |
| Add Unified Multimodal Prefix Cache (ablation) | ElasticMM with EMP · mixed-source request workload | normalized input token latency | Helped |
| Add non-blocking encoding (ablation) | ElasticMM with EMP and Unified Multimodal Prefix Cache · mixed-source request workload | normalized input token latency | Helped |
| Enable EMP without either multimodal inference optimization (ablation) | ElasticMM-EMP · mixed-source request workload | normalized input token latency | No clear effect |
Surprising findings
- Decoder-only models (Qwen2.5-VL) gain more from decoupling than encoder-decoder models, attributed to heavier prefill conflicting with image encoding.
Limitations
- Evaluated only on a single-node testbed with 8 A800 GPUs; multi-node scaling not studied.
- Only two models and two datasets evaluated.
Experimental setup
Datasets: VisualWebInstruct ShareGPT-4o
Benchmarks: VisualWebInstruct ShareGPT-4o
Metrics: TTFT normalized input latency normalized output latency maximum throughput under SLO P90 effective throughput
Baselines: vLLM DistServe ElasticMM static allocation variants (text-dominant, equal, multimodal-dominant)
Builds on: vLLM DistServe LoongServe Orca chunked prefill
Closest papers by meaning
- Efficiently Serving Large Multimodal Models Using EPD Disaggregation (ICML 2025 · similarity 0.89)
- Heterogeneous Parallelism for Multimodal Large Language Model Training (NeurIPS 2026 · similarity 0.88)
- EE-LLM: Large-Scale Training and Inference of Early-Exit Large Language Models with 3D Parallelism (ICML 2024 · similarity 0.86)
- TokenSwap: Benchmarking and Reducing the Modality Gap in Multimodal LLMs (NeurIPS 2026 · similarity 0.86)
- NaViL: Rethinking Scaling Properties of Native Multimodal Large Language Models under Data Constraints (NeurIPS 2025 · similarity 0.86)
- Recursive Inference Scaling: A Winning Path to Scalable Inference in Language and Multimodal Systems (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.