Research map / Multi-agent systems and games / LLM agents

CORRECT: COndensed eRror RECognition via knowledge Transfer in multi-agent systems

ICML 2026 · Multi-agent systems and games › LLM agents · approach: LLM multi-agent collaboration and safety · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes CORRECT, a training-free framework that distills recurrent multi-agent system failures into reusable error schemata cached and retrieved at inference to localize decisive errors, plus the CORRECT-Error synthetic benchmark of over 2,000 injected-error trajectories.

Problem

Localizing the earliest decisive error (agent and step) in failed multi-agent system trajectories is hard: LLM-as-a-judge is inaccurate on long logs, fine-tuning needs scarce labeled data, and retraining is impractical as new error types arise.

Approach

Distill failure trajectories into compact error schemata (signatures, context analysis, detection heuristics), cluster them to build a cache, and retrieve top-k schemata by embedding similarity to guide an LLM detector at inference, with online schema expansion and distillation.

Key results

• On Who&When Hand-Crafted, CORRECT raises GPT-5 exact-step accuracy from 8.6% to 17.2% and Qwen-2.5-7B from 3.5% to 12.1%.
• On Who&When Algorithm-Generated, GPT-5 + CORRECT reaches 38.1% vs 18.3% baseline (+19.8 points).
• On CORRECT-Error (GPT-4o-mini subset), CORRECT improves average Acc@1 by 20.1 points using Qwen-2.5-7b.

Surprising findings

• Naive in-context learning with retrieved full trajectories underperforms zero-shot baselines on error localization.
• Human labelers misclassified 47.1% of synthetic trajectories as human-labeled, near chance, while only 42.3% of genuine trajectories were correctly identified.

Limitations

• Decisive error labels may be subjective and ambiguous, especially with cascading failures.
• Retrieval relies only on semantic similarity; more structured retrieval is unexplored.
• Misfires on multi-error trajectories where a later error dominates the schema match.

Experimental setup

Datasets: Who&When CORRECT-Error HotpotQA Musique WikiMQA ARC MMLU-Pro Math500 GAIA

Benchmarks: Who&When CORRECT-Error AgentErrorBench

Metrics: step-level accuracy (Acc@0) tolerant accuracy (Acc@1, Acc@3, Acc@5)

Baselines: LLM-as-a-judge Fine-tuned Qwen-2.5-7B Naive ICL MIPRO

Builds on: Who&When MIPRO Magentic-One AutoGen

Closest papers by meaning

• Conformal Agent Error Attribution (NeurIPS 2026 · similarity 0.90)
• DoVer: Intervention-Driven Auto Debugging for LLM Multi-Agent Systems (ICLR 2026 · similarity 0.89)
• AgenTracer: Who Is Inducing Failure in the LLM Agentic Systems? (ICLR 2026 · similarity 0.89)
• AgentForesight: Online Auditing for Early Failure Prediction in Multi-Agent Systems (NeurIPS 2026 · similarity 0.89)
• Beyond LLM-Based Reasoning: Lightweight GNNs for Agent Failure Attribution (NeurIPS 2026 · similarity 0.89)
• Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systems (ICML 2025 · similarity 0.89)