AI Paper Atlas

Research map / Multi-agent systems and games / LLM agents

Aegis: Automated Error Generation and Identification for Multi-Agent Systems

Fanqi Kong, Ruijie Zhang, Huaxiao Yin, Guibin Zhang, Xiaofei Zhang, Ziang Chen, Zhaowei Zhang, Xiaoyuan Zhang, Song-Chun Zhu, Xue Feng

ICLR 2026 · Multi-agent systems and games › LLM agents · approach: LLM multi-agent decision making

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Aegis, a framework for automatically generating and identifying errors in LLM-based multi-agent systems to create large-scale error attribution datasets instead of costly manual annotation.

Problem

Multi-agent LLM systems are fragile and hard to debug, and large diverse datasets for error attribution are scarce because existing resources depend on costly, unscalable manual annotation.

Approach

Automates both the generation of errors in multi-agent system trajectories and the identification (attribution) of those errors, replacing manual annotation with scalable automated error injection and labeling.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
fine-tune with SFT on Aegis dataQwen2.5-14B-Instruct · Aegis-Bench + Who&When avg F1avg F1: 13.99 -> 26.51Improved
fine-tune with GRPO and hierarchical reward on Aegis dataQwen2.5-14B-Instruct · avg F1avg F1: 13.99 -> 18.41Improved
SFT on Aegis dataQwen2.5-7B-Instruct · Who&When Pairmicro-F1: 2.31 -> 1.26Hurt
SFT on Aegis dataQwen3-8B-Non-Thinking · avg F1avg F1: 10.12 -> 21.41Improved
GRPO on Aegis dataQwen3-8B-Thinking · avg F1avg F1: 13.06 -> 17.58Improved
remove hierarchical consistency loss (ablation)DCL · avg F1avg F1: 12.61 -> 9.52Hurt
remove text-prototype intent guidance (ablation)DCL · avg F1avg F1: 12.61 -> 10.16Hurt
use only-bilinear head instead of compositional head (ablation)DCL · avg F1avg F1: 12.61 -> 10.01Hurt
add chain-of-thought prompting to base open-source modelsQwen base models · Aegis error attributionavg F1Hurt
add chain-of-thought after GRPO trainingGRPO-trained Qwen models · Aegis error attributionavg F1Improved
train SFT beyond two epochs (ablation)Aegis-SFT · Who&Whenavg F1Hurt
SFT on Aegis for small MAS (Magentic-One, SmolAgents)Aegis-SFT · Aegis-Bench small MASavg scoreHurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy