Research map / Multi-agent systems and games / LLM agents
Scope Delineation Before Localization: A Two-Stage Framework for Enhancing Failure Attribution in Multi-Agent Systems
Kai Sun, Wenqiang Li, Bo Dong, Yuxin Lin, Jingyao Zhang, Bin Shi
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a two-stage framework that first delineates the scope of likely failure in multi-agent system logs before localizing the responsible agent and step, to improve LLM-based failure attribution.
Problem
Failure attribution in LLM-based multi-agent systems, where existing LLM methods struggle with lengthy interaction logs and lack expert debugging knowledge.
Approach
Drawing on human debugging strategies, it narrows the search scope within long logs before localizing the failure, rather than attributing directly over the full trace.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add two-stage scope delineation with Stepwise Scope Delineation before localization | STEP BY STEP · Who&When Hand-Crafted, GPT-4o | step accuracy (%): 8.77 -> 25.93 | Improved |
| add Expertise-Assisted Scope Delineation | SDBL · Who&When Hand-Crafted, GPT-4o | step accuracy (%): 25.93 -> 27.78 | Improved |
| use delineated scope (SSD) instead of random scope (RSD) (ablation) | SDBL · Who&When Hand-Crafted, GPT-4o | step accuracy (%): 14.81 -> 25.93 | Improved |
| use delineated scope (SSD) instead of random scope (RSD) (ablation) | SDBL · Who&When Algorithm-Generated, GPT-4o | step accuracy (%): 31.75 -> 32.54 | No clear effect |
| add Overstep paradigm to expertise-assisted scope (ablation) | SDBL (EASD) · Who&When Hand-Crafted, GPT-4o | step accuracy (%): 18.52 -> 27.78 | Improved |
| add Loop paradigm to expertise-assisted scope (ablation) | SDBL (EASD) · Who&When Hand-Crafted, GPT-4o | step accuracy (%): 12.96 -> 27.78 | Improved |
| apply SDBL (EASD) with DeepSeek-R1 backbone | ALL AT ONCE · Who&When Hand-Crafted, DeepSeek-R1 | agent accuracy (%): 55.56 -> 50.00 | Hurt |
| apply SDBL (EASD) with DeepSeek-V3 backbone | ALL AT ONCE · Who&When Algorithm-Generated, DeepSeek-V3 | step accuracy (%): 19.84 -> 39.68 | Improved |
| use SDBL (SSD) stepwise scope delineation instead of single-pass | SDBL · Who&When Hand-Crafted, GPT-4o | token consumption: 17106 -> 111310 | Hurt |
| use expertise-assisted scope in a single pass | SDBL (EASD) · Who&When Hand-Crafted, GPT-4o | token consumption: 17106 -> 24657 | Mixed |
| prompt LLM directly for K failure-prone steps (ALL AT ONCE*) instead of delineation (ablation) | scope delineation · Who&When Hand-Crafted Hit@K | Hit@K (%): 9.26 -> 25.93 (Hit@3 vs EASD) | Hurt |
Closest papers by meaning
- Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systems (ICML 2025 · similarity 0.90)
- Beyond LLM-Based Reasoning: Lightweight GNNs for Agent Failure Attribution (NeurIPS 2026 · similarity 0.90)
- Who&When Pro: Can LLMs Really Attribute Failures in AI Agents? (NeurIPS 2026 · similarity 0.90)
- DoVer: Intervention-Driven Auto Debugging for LLM Multi-Agent Systems (ICLR 2026 · similarity 0.89)
- CORRECT: COndensed eRror RECognition via knowledge Transfer in multi-agent systems (ICML 2026 · similarity 0.88)
- AgenTracer: Who Is Inducing Failure in the LLM Agentic Systems? (ICLR 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
