Research map / Multi-agent systems and games / LLM agents

When Embedding-Based Defenses Fail: Rethinking Safety in LLM-Based Multi-Agent Systems

ICML 2026 · Multi-agent systems and games › LLM agents · approach: LLM multi-agent decision making · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows embedding-based MAS defenses fail under near-benign attacks (Slow Drift, Benign Wrapper, Chaos Seeding) and proposes token-confidence-based pruning and down-weighting as a complementary defense.

Problem

Embedding-based defenses that detect and prune malicious agents in LLM multi-agent systems rely on malicious and benign message embeddings being separable; attackers can craft near-benign messages to evade them.

Approach

Theoretically show that a benign margin forces a broad benign acceptance region in embedding space, so near-benign attack embeddings evade thresholded detectors; propose three near-benign attacks and use token-level entropy-based confidence (pruning or down-weighting messages) as a model-internal signal that remains informative when embeddings are not separable.

Key results

• Under near-benign attacks on LLaMA-3.1-8B, confidence-guided defense reaches Acc-Avg 52.7 on MMLU vs 39.3 for G-Safeguard and 32.6 for GUARDIAN; on GSM8K 77.7 vs 66.7 and 46.2.
• On GPT-4o-mini, confidence-guided defense achieves Acc-Avg 71.9 on MMLU vs 64.8 (G-Safeguard) and 61.0 (GUARDIAN) and 64.5 with no defense.
• Denser communication topologies shorten the signal half-life: fully connected has half-life 1 and ASR@R 0.48 versus 0.18 for chain.

Surprising findings

• Embedding-based defenses sometimes perform worse than no defense under near-benign attacks (e.g., G-Safeguard on BBH with Benign Wrapper), as they prune benign messages.
• Under Chaos Seeding, benign-benign embedding distances for different-answer pairs can exceed benign-attacker distances, so disagreement dominates embedding geometry.

Limitations

• Confidence signal itself decays after sustained contamination when discussion converges to noisy generations.
• GPT-4o-mini confidence is derived from returned token probabilities rather than full logits.
• Evaluation limited to three multiple-choice/math reasoning benchmarks and three models with subsampled test sets.

Experimental setup

Datasets: MMLU GSM8K BBH

Benchmarks: MMLU GSM8K BBH

Metrics: accuracy majority-vote accuracy exact-match accuracy attack success rate AUROC cosine distance

Baselines: No defense G-Safeguard GUARDIAN

Builds on: G-Safeguard GUARDIAN BlindGuard Sentence-BERT

Closest papers by meaning

• Attack the Messages, Not the Agents: A Multi-round Adaptive Stealthy Tampering Framework for LLM-MAS (AAAI 2026 · similarity 0.90)
• Securing Multi-Agent Systems Against Corruptions via Node Contribution Backpropagation (ICML 2026 · similarity 0.90)
• Let Models Speak Ciphers: Multiagent Debate through Embeddings (ICLR 2024 · similarity 0.89)
• Cut the Crap: An Economical Communication Pipeline for LLM-based Multi-Agent Systems (ICLR 2025 · similarity 0.89)
• Shadows in the Code: Exploring the Risks and Defenses of LLM-based Multi-Agent Software Development Systems (AAAI 2026 · similarity 0.88)
• SafeSieve: From Heuristics to Experience in Progressive Pruning for LLM-based Multi-Agent Communication (AAAI 2026 · similarity 0.88)