AI Paper Atlas

Research map / Large language models / Agents, code and math

ThunderAgent: A Fast, Simple, and Program-Aware Agentic Inference System

Hao Kang, Ziyang li, Weili Xu, Xinyu Yang, Yinfang Chen, Junxiong Wang, Beidi Chen, Tushar Krishna, Chenfeng Xu, Simran Arora

ICML 2026 · Large language models › Agents, code and math · approach: Agent instruction-following benchmarks

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use program-aware scheduler with state-aware pausing instead of request-level eviction (vLLM)ThunderAgent · serving, 96 parallel programsthroughput: 1.48-3.58x over vLLMImproved
use time-decay function f(t) rather than static TTL pinning for acting programs (ablation)ThunderAgent · stochastic tool execution (ToolOrchestra on HLE)throughputImproved
replace session-based node pinning with global program-aware waiting queue (ablation)ThunderAgent · distributed rollout, 2 to 8 nodesspeedup over SGLang cache-aware routing: 1.79x at 2 nodes -> 2.39x at 8 nodesImproved
add hook-based garbage collection and asynchronous environment preparation (ablation)ThunderAgent · OpenHands rolloutdisk memory and latency: 4.2x disk savings, about 10% latency improvementImproved
increase detection period Delta t (ablation)ThunderAgent · mini-SWEAgent, GLM4.6, single H100throughputHurt
increase base x of decay f(t)=x^-t (more aggressive eviction) (ablation)ThunderAgent · mini-SWEAgent, GLM4.6, single H100throughputHurt
vary Delta t and decay base across settings (ablation)ThunderAgent · mini-SWEAgent, GLM4.6throughputNo clear effect
add KV cache offloading (vLLM + LMCache) as a remedy for memory pressurevLLM · GLM-4.6 with mini-SWEAgentthroughputHurt
apply prefill-decode disaggregation to agentic workloadsvLLM · agentic servingthrashing / throughputHurt
add ThunderAgent on top of SGLang HiCache offloadingSGLang HiCache · MiniMax M2.5 on R2E-Gym, 8xH100, batch 64-192throughput and latency: P50 latency flat at ~10 s vs ~30 s at batch 160 and ~70 s at batch 192Improved
run ThunderAgent on A100 at low concurrency (24)ThunderAgent · 8xA100, mini-SWEAgent and OpenHandsthroughputNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy