Research map / Multi-agent systems and games / LLM agents

RoutingBench: Can Agentic Routing Analysis Scale to Production Datacenter Networks?

NeurIPS 2026 · Multi-agent systems and games › LLM agents · approach: Software engineering agent benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces RoutingBench, an automatically generated benchmark for LLM agent routing-path analysis on fat-tree datacenter and inter-datacenter networks, and shows that an agent-skills approach with GPT-5.4 scales to 50K routers at 99.5% accuracy.

Problem

Routing-path analysis in hyperscale networks is a scalability bottleneck for symbolic tools like Batfish, which run out of memory beyond about 8K routers; it is unclear whether agentic LLM approaches can scale it while staying accurate.

Approach

Compares three agent paradigms (in-context learning, iterative reasoning over raw configs, and agent skills that extract task-relevant config evidence via tools) and proposes the "explore more; digest less" principle, where agents invoke skills and tools rather than reading raw configurations.

Key results

• GPT-5.4 with Agent Skills reaches 99.5% accuracy on intra-datacenter single changes at 50K routers (k=200) using under 120K tokens.
• Batfish runs out of memory beyond 8K routers, while In-Context Learning overflows the context window at 500 routers for GPT-5.4.
• Agent Skills accuracy drops to about 73.5% on inter-datacenter networks at 100K routers and about 79% on compound changes at 50K routers.

Surprising findings

• Thinking-oriented models (o3, DeepSeek-V4-Pro, Qwen-3.5) use roughly 3x more tokens than GPT-5.4 to reach similar accuracy.
• Compound changes involving multiple protocols on different routers are much harder, with accuracy falling to about 71-73% at 50K routers, while most individual changes stay near 100%.

Limitations

• Benchmark covers only routing configurations; production configs also include bandwidth, authentication and other complexity.
• Ground truth beyond Batfish's capacity (~8K routers) is derived analytically, not by simulation.
• Accuracy degrades on inter-datacenter and compound changes at scale, so the approach is not yet reliable there.

Experimental setup

Datasets: synthetically generated fat-tree and inter-datacenter network configurations

Benchmarks: RoutingBench

Metrics: accuracy scalability (max k) tokens runtime LLM rounds tool invocations

Baselines: In-Context Learning Iterative Reasoning Batfish GPT-4o OpenAI o3 DeepSeek-V4-Pro Qwen-3.5-397B-A17B Llama-3.3-70B-Instruct

Builds on: Batfish chain-of-thought prompting ReAct

Closest papers by meaning

• SkillOrchestra: Learning to Route Agents via Skill Transfer (NeurIPS 2026 · similarity 0.86)
• Solving Multiagent Path Finding on Highly Centralized Networks (AAAI 2025 · similarity 0.84)
• MonoScale: Scaling Multi-Agent System with Monotonic Improvement (ICML 2026 · similarity 0.84)
• Routers Learn the Geometry of Their Experts: Geometric Coupling in Sparse Mixture-of-Experts (NeurIPS 2026 · similarity 0.83)
• The Scaling Laws of Skills in LLM Agent Systems (NeurIPS 2026 · similarity 0.83)
• Augmented Runtime Collaboration for Self-Organizing Multi-Agent Systems: A Hybrid Bi-Criteria Routing Approach (AAAI 2026 · similarity 0.83)