Research map / Multi-agent systems and games / LLM agents

The Scaling Laws of Skills in LLM Agent Systems

NeurIPS 2026 · Multi-agent systems and games › LLM agents · approach: Robust and efficient agent execution · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Studies how growing LLM skill libraries affect routing accuracy and downstream execution across 15 frontier LLMs, proposing logarithmic routing and multiplicative/rescue execution laws linked by a decay slope, and a law-guided skill manager that improves routing and transfers to ClawBench/ClawMark.

Problem

Skill libraries for LLM agents grow large, but how routing and execution performance scale with library size and structure is poorly understood; wrong in-library selection (hijack) is the main failure mode.

Approach

Empirical laws: single-step routing accuracy decays as a - b ln N; execution gives multiplicative joint routing before state and a rescue effect after correct execution; a single slope b links routing collapse to execution rescue across models without refitting. These laws guide library edits (neighbor auditing, boundary rewriting, abstract-skill removal, prompt anchoring).

Key results

• Single-step routing accuracy fits a - b ln N with R^2 > 0.97 for all 15 models; the cross-model prediction of execution rescue from b reaches rho = 0.74 (p < 0.001).
• Law-guided skill manager raises held-out routing accuracy from 71.3% to 91.7% and reduces hijack from 22.4% to 4.1% on 1,600 held-out tasks.
• Downstream mean pass rate improves from 49.3% to 61.6% on ClawBench and from 28.4% to 34.5% on ClawMark.

Surprising findings

• Vague prompts yield 0% hallucinated skill names while task-level routing accuracy falls to 18.3% with in-library hijack above 80%.
• Loose-dependency pairs gain 2.8% when wrong upstream output is present, because the model ignores the bad output.

Limitations

• Laws are empirical for flat natural-language skill libraries and availability-preserving settings; may not hold for learned routers, fine-tuning, retrieval, or hierarchical libraries.
• Several derivations rely on assumptions and conditional/local approximations (e.g., f(N) proportional to ln N, Gaussian noise independence).
• Task synthesis used GPT-5.4 assistance and the library is curated, which may limit external validity.

Experimental setup

Datasets: 1,141 real-world agent skills (Agent Skills, Claude Code, MCP, GitHub) Terminal-Bench τ-Bench τ²-Bench FlowBench PinchBench

Benchmarks: ClawBench ClawMark

Metrics: routing accuracy hijack rate hallucination rate pass rate R^2 Spearman correlation

Baselines: ComplexFuncBench BiasBusters xRouter EvoRoute AutoTool MetaToolAgent

Builds on: ReAct Toolformer ToolLLM

Closest papers by meaning

• Evaluating Test-Time Scaling of General LLM Agents (NeurIPS 2026 · similarity 0.88)
• SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks (NeurIPS 2026 · similarity 0.88)
• PROTEUS: A Self-Evolving Red Team with Surface Expansion for Agent Skill Ecosystems (NeurIPS 2026 · similarity 0.87)
• UltraHorizon: Benchmarking LLM-Agent Capabilities in Ultra Long-Horizon Scenarios (ICML 2026 · similarity 0.87)
• Automata from Agent Traces: Failure and Next-Step Prediction (NeurIPS 2026 · similarity 0.87)
• MonoScale: Scaling Multi-Agent System with Monotonic Improvement (ICML 2026 · similarity 0.87)