Research map / Multi-agent systems and games / LLM agents
Attractive Metadata Attack: Inducing LLM Agents to Invoke Malicious Tools
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Attractive Metadata Attack (AMA), a black-box in-context optimization that crafts attractive but valid tool metadata so LLM agents preferentially invoke a malicious tool, evaluated on ten simulated scenarios across four LLM agents.
Problem
LLM agents choose tools based on metadata (names, descriptions, parameter schemas), which opens a stealthy attack surface where malicious tools get selected without prompt injection or access to model internals.
Approach
Treats malicious tool metadata generation as a state-action-value optimization solved via LLM in-context learning, using generation traceability, weighted value evaluation (absolute invocation probability plus relative improvement over parent), and batch generation with top-K state updates.
Key results
- AMA achieves 81%-95% attack success rate in targeted settings across four agents on ten simulated scenarios.
- Combining AMA with injection attacks raises ASR to roughly 94-99% across models.
- Under the Rewrite and Refuge defenses, AMA retains high ASR in targeted settings (e.g., 89.00% on Gemma3-27B with Refuge), while the auditor's TPR on AMA tools is far below the Injection Attack's 55%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use AMA-optimized tool metadata instead of Prompt Attack metadata | LLM agents · Targeted tool-use scenarios | ASR: 83.60 -> 95.58 | Helped |
| Use AMA-optimized tool metadata instead of Injection Attack | LLaMA3.3-70B · Targeted tool-use scenarios | ASR: 75.80 -> 94.80 | Helped |
| Use AMA-optimized tool metadata instead of Injection Attack | GPT-4o-mini · Targeted tool-use scenarios | ASR: 89.00 -> 81.43 | Hurt |
| Combine AMA with Injection Attack | Gemma3-27B · Targeted tool-use scenarios | ASR: 95.58 -> 95.33 | Hurt |
| Combine AMA with Injection Attack | LLM agents · Untargeted tool-use scenarios | ASR: 83.10 -> 99.20 | Helped |
| Add Prompt Refuge defense | AMA · Gemma3-27B targeted | ASR: 95.58 -> 89.00 | Hurt |
| Add Dynamic Prompt Rewriting defense | AMA · Gemma3-27B targeted | ASR: 95.58 -> 90.50 | Hurt |
| Transfer optimized metadata across tools within the same domain | AMA · Gemma3-27B cross-tool transfer | ASR: 33.33 -> 90.83 | Helped |
| Transfer optimized metadata to a different domain | AMA · GPT-4o-mini cross-tool transfer | ASR: 65.71 -> 2.86 | Hurt |
| Vary declared tool parameter count from 0 to 10 (ablation) | AMA · Tool-use scenarios | ASR | No clear effect |
| Set weighting coefficient λ to 0.5 instead of λ = 0 (ablation) | AMA optimization · Malicious tool generation | Selection probability | Helped |
Surprising findings
- Cross-domain transfer of optimized metadata sharply reduces ASR (e.g., GPT-4o-mini drops to 2.9%), indicating effectiveness depends on domain semantic alignment.
- Adding more declared tool parameters does not consistently reduce attack effectiveness, contrary to intuition.
Limitations
- Assumes agents select tools primarily by metadata, ignoring provider credibility or reputation signals.
- Evaluated in simulated tool-use environments with simulated APIs rather than real marketplaces.
- Untargeted attacks are substantially weaker on some models (e.g., Qwen2.5-32B ASR about 39% without defense).
Experimental setup
Datasets: ASB ToolBench AI4Privacy
Benchmarks: ASB AgentBench
Metrics: Task Completion Attack Success Rate Parameter Response Privacy Leakage TPR FPR
Baselines: Injection Attack Prompt Attack Dynamic Prompt Rewriting Prompt Refuge
Builds on: ReAct ToolBench AgentBench ASB
Closest papers by meaning
- Think Twice Before You Act: Protecting LLM Agents Against Tool Description Poisoning via Isolated Planning (ICML 2026 · similarity 0.90)
- MCPTox: A Benchmark for Tool Poisoning on Real-World MCP Servers (AAAI 2026 · similarity 0.88)
- Skill-Inject: Measuring Agent Vulnerability to Skill File Attacks (NeurIPS 2026 · similarity 0.88)
- Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents (ICLR 2025 · similarity 0.88)
- Memory Injection Attacks on LLM Agents via Query-Only Interaction (NeurIPS 2025 · similarity 0.88)
- A-MemGuard: A Proactive Defense Framework For LLM-Based Agent Memory (ICML 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.