Research map / Multi-agent systems and games / LLM agents

Attack the Messages, Not the Agents: A Multi-round Adaptive Stealthy Tampering Framework for LLM-MAS

AAAI 2026 · Multi-agent systems and games › LLM agents · approach: Agent safety and attack benchmarks · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MAST, a multi-round adaptive stealthy message-tampering attack on LLM multi-agent systems that trains an attack policy with MCTS-derived step-level DPO and constrains edits with semantic and embedding similarity thresholds.

Problem

LLM multi-agent systems communicate through messages that can be intercepted and tampered with; existing attacks either compromise agent internals, use overt persuasion, or rely on task-specific templates, limiting effectiveness, adaptability and stealthiness.

Approach

Train an attack policy LLM via MCTS over sub-goal sequences with a process reward model, convert node value gaps into step-level preference pairs for DPO, and then execute tampering under dual semantic and embedding similarity constraints so that edits look benign.

Key results

• MAST reaches ASR of 85.4 (MAB.code), 88.3 (MAB.research) and 95.3 on hierarchical MAB.code in the Flat/Hier settings, versus at most 76.5 for the best baseline (AiTM-Target).
• Baselines like AutoInject reach high stealthiness (e.g., 76.9) but low ASR (31.1 on MAB.code Flat), while AiTM variants show high ASR but low stealthiness (e.g., 28.8 to 36.1).
• Fine-tuned Qwen3-8B reaches 84.5 ASR on MAB versus 63.7 for GPT-4o and 56.4 for Gemini 2.5 Pro.

Surprising findings

• The untrained tampering mechanism already shows relatively high stealthiness due to the dual constraints.
• Removing the training stage causes ASR to collapse to about 32 on MAB, showing the trained policy is critical.

Limitations

• Stealthiness is evaluated only against an LLM-based tamper defender that the authors also configure, not against a real deployed security system.
• ASR is defined as the attacker's goal appearing in the final output, which is a proxy measure, and the paper does not report human evaluation of stealthiness.
• The process reward model relies on GPT-4o and the attack policy requires per-task MCTS simulation of the multi-agent system, which adds cost.

Experimental setup

Datasets: MultiAgentBench HumanEval MMLU

Benchmarks: MultiAgentBench code MultiAgentBench research HumanEval MMLU physics MMLU biology MMLU math

Metrics: attack success rate stealthiness

Baselines: Debate-Attack AutoInject AiTM-Target AiTM-Dos

Builds on: Direct Preference Optimization Monte Carlo Tree Search AiTM

Closest papers by meaning

• When Embedding-Based Defenses Fail: Rethinking Safety in LLM-Based Multi-Agent Systems (ICML 2026 · similarity 0.90)
• When Agents Go Rogue: Activation-Based Detection of Malicious Behaviors in Multi-Agent Systems (ICML 2026 · similarity 0.88)
• DRIFT: Dynamic Rule-Based Defense with Injection Isolation for Securing LLM Agents (NeurIPS 2025 · similarity 0.88)
• Securing Multi-Agent Systems Against Corruptions via Node Contribution Backpropagation (ICML 2026 · similarity 0.88)
• Attractive Metadata Attack: Inducing LLM Agents to Invoke Malicious Tools (NeurIPS 2025 · similarity 0.87)
• To trust or not to trust: Attention-based Trust Management for LLM Multi-Agent Systems (NeurIPS 2026 · similarity 0.87)