Research map / Multi-agent systems and games / LLM agents

To trust or not to trust: Attention-based Trust Management for LLM Multi-Agent Systems

NeurIPS 2026 · Multi-agent systems and games › LLM agents · approach: LLM multi-agent decision making · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes six trust dimensions for inter-agent messages, a logistic-regression trust scorer (A-Trust) over LLM attention head patterns, and a trust management system that filters messages and flags malicious agents in LLM multi-agent systems.

Problem

LLM-based multi-agent systems accept incoming messages without assessing trustworthiness, making them vulnerable to deceptive or malicious inter-agent messages and communication attacks.

Approach

Defines six orthogonal trust dimensions (factual accuracy, logical consistency, relevance, bias, clarity, quality) and shows that per-head attention statistics from an open LLM can be linearly probed to score each dimension, with thresholded message filtering and agent-level violation-rate records.

Key results

• A-Trust raises message detection rate to roughly 80-90% across datasets and structures, versus about 50-60% for perplexity and prompt baselines (Chain, MMLUPhy: 84.3 vs 52.9 for prompt).
• On MMLUPhy Complete structure, A-Trust cuts AiTM attack success from 94.6% to 23.5% relative to no trust.
• Agent-level trust records achieve 100% agent detection rate across all evaluated attacks, with clean-run clean error increase under 2 points.

Surprising findings

• Prompt-based trust scores from GPT-4o sometimes fail to distinguish trust violations that attention patterns separate clearly.
• Perplexity-based scores perform near chance for some attacks (e.g., 43.1% MDR for AiTM on MMLUPhy Chain).

Limitations

• Covers only six trust dimensions and malicious violations; other misalignment sources such as poor coordination are not addressed.
• Assumes balanced resource allocation across agents.
• Factual dimension relies on external verifier (Gemini-2.0-flash with Google Search) for uncertain cases; the internal probe is weaker for facts.

Experimental setup

Datasets: Trust Violation (new) FEVER StereoSet MMLU physics subset StrategyQA MATH-500 MBPP SoftwareDev

Benchmarks: MMLU physics StrategyQA MATH-500 MBPP SoftwareDev

Metrics: message detection rate agent detection rate attack success rate clean error AUC-style density separation

Baselines: perplexity score prompt-based trust score No trust

Builds on: Camel MetaGPT AiTM NetSafe AutoTransform/AutoInject

Closest papers by meaning

• Position: TrustLLM: Trustworthiness in Large Language Models (ICML 2024 · similarity 0.89)
• Can Large Language Model Agents Simulate Human Trust Behavior? (NeurIPS 2024 · similarity 0.88)
• When Embedding-Based Defenses Fail: Rethinking Safety in LLM-Based Multi-Agent Systems (ICML 2026 · similarity 0.87)
• Attack the Messages, Not the Agents: A Multi-round Adaptive Stealthy Tampering Framework for LLM-MAS (AAAI 2026 · similarity 0.87)
• U-trustworthy Models. Reliability, Competence, and Confidence in Decision-Making (AAAI 2024 · similarity 0.87)
• OpenDeception: Learning Deception and Trust in Human–AI Interaction via Multi-Agent Simulation (ICML 2026 · similarity 0.86)