Research map / Multi-agent systems and games / LLM agents

SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks

NeurIPS 2026 · Multi-agent systems and games › LLM agents · approach: Robust and efficient agent execution · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces SkillsBench, a paired benchmark of 87 expertise-heavy tasks that measures how much curated agent Skills improve 18 model-harness configurations versus no-Skills runs (+16.6 pp average).

Problem

There is no standard way to measure whether inference-time Agent Skills (procedural knowledge packages for LLM agents) actually help agents on expertise-heavy tasks, as opposed to folding augmentation effects into raw model capability.

Approach

A paired evaluation protocol that runs each containerized task under matched no-Skills, curated-Skills (and self-generated-Skills on a subset) conditions with deterministic verifiers, so Skill effects are isolated from model-harness configuration; tasks are sourced from vetted contributors with leakage gates that forbid Skills containing task-specific answers.

Key results

• Curated Skills raise average pass rate from 33.9% to 50.5% (+16.6 pp; 25.5% normalized gain) across 18 model-harness configurations.
• Configuration-level gains range from +4.1 pp (Gemini 3.1 Flash Lite) to +25.7 pp (OpenHands + GLM 5.1).
• Self-generated Skills fall below the no-Skills baseline on all three dedicated-harness configurations (-8.1 to -11.5 pp).

Surprising findings

• Self-generated Skills reduce performance below the no-Skills baseline on all three dedicated-harness configurations.
• Comprehensive documentation yields essentially no gain (+0.7 pp) while focused skills yield +19 to +21.5 pp.

Limitations

• Focuses on terminal-based containerized tasks; may not transfer to GUI, multi-agent, or very long-horizon settings.
• Gains may partly reflect longer context rather than procedural structure; lacks length-matched controls.
• Benchmark Skills are high-quality (mean 10.1/12) versus ecosystem mean 6.2/12, an optimistic scenario.

Experimental setup

Datasets: SkillsBench tasks (87 curated tasks from 400 contributor submissions) Skills ecosystem snapshot (2,014,000 Skills)

Benchmarks: SkillsBench

Metrics: task-macro pass rate absolute pass-rate gain (pp) normalized gain

Baselines: no-Skills condition self-generated Skills condition

Builds on: Terminal-Bench BenchFlow Harbor task format Agent Skills format

Closest papers by meaning

• From Raw Experience to Skill Consumption: A Systematic Study of Model-Generated Agent Skills (NeurIPS 2026 · similarity 0.91)
• SkillMaster: Toward Autonomous Skill Mastery in LLM Agents (NeurIPS 2026 · similarity 0.91)
• SkillOS: Learning Skill Curation for Self-Evolving Agents (NeurIPS 2026 · similarity 0.89)
• Skill-Pro: Learning Reusable Skills from Experience via Non-Parametric PPO for LLM Agents (ICML 2026 · similarity 0.89)
• AgentBench: Evaluating LLMs as Agents (ICLR 2024 · similarity 0.89)
• Group of Skills: Group-Structured Skill Retrieval for Agent Skill Libraries (NeurIPS 2026 · similarity 0.89)