Research map / Multi-agent systems and games / LLM agents

ScaleSim: Serving Large-Scale Multi-Agent Simulation with Invocation Distance-Based Memory Management

ICML 2026 · Multi-agent systems and games › LLM agents · approach: Robust and efficient agent execution

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ScaleSim, an SGLang-based LLM serving system that uses an "invocation distance" estimate per agent to drive proactive prefetching and future-reuse-aware eviction of agent-specific GPU memory (LoRA adapters, prefix caches) in large-scale multi-agent simulations.

Problem

Large-scale LLM-based multi-agent simulations exhaust GPU memory because each agent keeps private GPU-resident states (models, prefix caches, adapters), causing reactive CPU-GPU swaps and poor eviction decisions under LRU-style policies.

Approach

Exploits sparse agent activation and estimable invocation order by defining a relative per-agent "invocation distance" (action remaining time, interaction distance, or hop count) exposed via a unified interface, then uses it for distance-guided prefetching, eviction, and preemptive load scheduling, with a modular agent-specific memory abstraction.

Key results

• Up to 1.73x speedup over SGLang on AgentSociety (independent simulation).
• Up to 1.31x speedup on Generative Agents and up to 1.74x on information diffusion.
• Reduces TTFT by 48%-68% compared to HiCache under high agent concurrency in AgentSociety.

Surprising findings

• Overall speedup is bounded by the Decode phase share, since prefill and decode are unaffected by the memory management policy.

Limitations

• Relies on application-provided invocation distance estimates, which may be inaccurate; preemption is a fallback for mispredictions.
• Agent-specific memory in experiments is simulated with a shared LoRA adapter and distinct logical IDs rather than truly distinct adapters.
• Benefits are reduced under dense activation (moderate sparsity).

Experimental setup

Datasets: AgentSociety Generative Agents

Benchmarks: AgentSociety Generative Agents information diffusion simulation

Metrics: end-to-end speedup time-to-first-token per-step execution time execution time breakdown

Baselines: SGLang HiCache

Builds on: SGLang S-LoRA HiCache

Closest papers by meaning

• Evaluating Test-Time Scaling of General LLM Agents (NeurIPS 2026 · similarity 0.90)
• KVFlow: Efficient Prefix Caching for Accelerating LLM-Based Multi-Agent Workflows (NeurIPS 2025 · similarity 0.88)
• CONCUR: High-Throughput Agentic Batch Inference of LLM via Congestion-Based Concurrency Control (ICML 2026 · similarity 0.88)
• MonoScale: Scaling Multi-Agent System with Monotonic Improvement (ICML 2026 · similarity 0.87)
• Justitia: Fair and Efficient Scheduling of Task-parallel LLM Agents with Selective Pampering (NeurIPS 2026 · similarity 0.87)
• A Two-Tier Perspective on Inference-Time Parallelism in Multi-Agent LLM Systems (ICML 2026 · similarity 0.87)