AI Paper Atlas

Research map / Multi-agent systems and games / LLM agents

DDBench: A Benchmark for Agentic Debugging on Distributed Systems

Yibo Yan, Huijuan Wang, Junzhou He, Yizhuo Liang, Shaoyu Wang, Huanchen Sun, Seo Jin Park

NeurIPS 2026 · Multi-agent systems and games › LLM agents · approach: Robust and efficient agent execution · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces DDBench, a benchmark for evaluating LLM coding agents on debugging bugs in distributed systems, where root causes span processes, nodes, and protocol interactions.

Problem

LLM coding agents perform well on single-process SWE tasks, but debugging distributed systems remains under-explored because bugs span processes and nodes, root causes are rarely recoverable from source alone, and non-deterministic interleavings make brute-force exploration intractable.

Approach

A benchmark targeting agentic debugging of distributed-system bugs, distinct from single-process SWE-bench-style tasks; the abstract indicates it addresses cross-process, cross-node, and protocol-level failures. Specific method details are not stated in the abstract.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add debug-context bundle (logs, traces, notes) (ablation)10 LLMs with mini-swe-agent · DDBench tier-1aggregate pass rate: 31.0% -> 50.0% (+19.0 pp)Improved
add debug-context bundle (ablation)GLM 4.7 · DDBench tier-1pass rate: 9.7% -> 48.4%Improved
add debug-context bundle (ablation)Claude Opus 4.6 · DDBench tier-1pass rate: 51.6 -> 64.5 (p=0.13)No clear effect
add debug-context bundle (ablation)Claude Opus 4.6 · DDBench tier-1tokens per case: 3.11M -> 1.21M (-61%)Improved
add debug-context bundle (ablation)10 LLMs · DDBench tier-1mean steps: 57.6 -> 43.1Improved
add debug-context bundle (ablation)10 LLMs · DDBench tier-2aggregate pass rate: 48.0% -> 62.7%Improved
add debug-context bundle (ablation)10 LLMs · DDBench tier-3aggregate pass rate: 67.9% -> 77.1%; two models -7.1 ppMixed
add debug-context bundle (ablation)10 LLMs · DDBench tier-1 concurrency hazardspass rate: 30.0% -> 75.0%Improved
add debug-context bundle (ablation)10 LLMs · DDBench tier-1 protocol and recoverypass rate: 27.7% -> 39.2%Improved
provide debug-context far from the fault (large abstraction distance) (ablation)Claude Opus 4.6 · Kvrocks-3356 casedebugging pathHurt
move from SWE-bench Verified to DDBench tier-1 for model comparisonsix frontier models · pass rate spreadspread across models: 7 pp -> 42 ppImproved

Experimental setup

Benchmarks: DDBench

Builds on: SWE-bench Verified

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy