Research map / Multi-agent systems and games / LLM agents

Goal-Aware Identification and Rectification of Misinformation in Multi-Agent Systems

ICLR 2026 · Multi-agent systems and games › LLM agents · approach: Self-evolving language agent learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces the MisinfoTask dataset for red-teaming misinformation in LLM multi-agent systems and ARGUS, a training-free defense that localizes critical communication channels and uses goal-aware CoT rectification to correct misinformation.

Problem

LLM-based multi-agent systems are vulnerable to covert misinformation injected via prompts, RAG databases, or tools, which can propagate through agent communication and cause task failure; existing datasets and defenses do not target covert misinformation on complex tasks.

Approach

A training-free two-stage defense: betweenness-centrality and adaptive semantic-relevance edge monitoring places a corrective agent on critical channels, which uses chain-of-thought to detect misinformation, rectify it, and infer the attacker's goal to guide subsequent-round monitoring.

Key results

• ARGUS reduces average MT from 5.22 to 3.43 on GPT-4o-mini and to 3.25 on DeepSeek-V3 under attack, with average MT reduction around 28% across injection methods (as reported in the text).
• Under attack without defense, average TSR falls to 67-81% across core LLMs; ARGUS raises TSR for DeepSeek-V3 from 80.72% to 84.33%.
• Ablation: removing CoT revision or dynamic localization degrades ARGUS; supplying ground truth to the corrective agent lowers MT further (e.g., PI 3.317 vs 3.502 for full ARGUS).

Surprising findings

• Self-Check and G-Safeguard provide limited gains, and G-Safeguard sometimes increases MT under RAG poisoning (e.g., 5.19 vs 4.95 for GPT-4o-mini).

Limitations

• Added defense module introduces computational overhead.
• Only addresses misinformation about knowledge stored in core LLM parameters, not time-sensitive external information.
• Dataset generated by GPT-4o and manually filtered to 108 instances; limited scale.

Experimental setup

Datasets: MisinfoTask

Benchmarks: MisinfoTask

Metrics: Misinformation Toxicity (MT) Task Success Rate (TSR)

Baselines: Self-Check G-Safeguard

Builds on: Self-Check G-Safeguard ReAct Chain-of-Thought

Closest papers by meaning

• Hierarchical Attacks for Multi-Modal Multi-Agent Reasoning (CVPR 2026 · similarity 0.86)
• When Embedding-Based Defenses Fail: Rethinking Safety in LLM-Based Multi-Agent Systems (ICML 2026 · similarity 0.86)
• T2MAC: Targeted and Trusted Multi-Agent Communication through Selective Engagement and Evidence-Driven Integration (AAAI 2024 · similarity 0.85)
• Attack the Messages, Not the Agents: A Multi-round Adaptive Stealthy Tampering Framework for LLM-MAS (AAAI 2026 · similarity 0.84)
• Systematic Failures in Collective Reasoning under Distributed Information in Multi-Agent LLMs (ICML 2026 · similarity 0.84)
• AgentCollabBench: Diagnosing When Good Agents Make Bad Collaborators (NeurIPS 2026 · similarity 0.84)