Research map / Adversarial robustness and security / Attacks on text and graphs
Learning to Lie: Adversarial Attacks on Human-AI Teams and LLMs
Abed K. Musaffar, Anand Gokhale, Sirui Zeng, Rasta Tadayon, Xifeng Yan, Ambuj Singh, Francesco Bullo
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use cognitive-model MBRL attacker (lies) in last 15 rounds | human-AI teams · trivia team score | cumulative score: 15% reduction | Hurt |
| use MLP-model MBRL attacker (lies) in last 15 rounds | human-AI teams · trivia team score | cumulative score: 24% reduction | Hurt |
| use MLP attacker instead of cognitive attacker | MBRL attack · trivia team score | team score: p < 0.05 | Improved |
| cognitive-model attack versus no-attack | MBRL attack · trivia team score | team score: p = 0.12 | No clear effect |
| add current performance (c.p.) to influence models (ablation) | cognitive and MLP models · score prediction | MSE | Improved |
| add current performance (c.p.) to influence models (ablation) | influence matrix prediction models · influence matrix prediction | MSE | No clear effect |
| vary attacker window size from 0 to 24 rounds (ablation) | MLP and cognitive attacker · simulated cumulative final error | cumulative final error | No clear effect |
| add chat logs to LLM input (ablation) | ChatGPT 4o-mini influence allocation · influence allocation | performance | Improved |
| add full performance history to LLM input (ablation) | ChatGPT 4o-mini influence allocation · influence allocation | performance | No clear effect |
| use chain-of-thought reasoning LLMs (o3-mini, DeepSeek-R1) versus non-reasoning | LLM team members · influence allocation under attack | performance | Mixed |
Closest papers by meaning
- OpenDeception: Learning Deception and Trust in Human–AI Interaction via Multi-Agent Simulation (ICML 2026 · similarity 0.89)
- LLM Agents Can Be Choice-Supportive Biased Evaluators: An Empirical Study (AAAI 2025 · similarity 0.87)
- On Targeted Manipulation and Deception when Optimizing LLMs for User Feedback (ICLR 2025 · similarity 0.87)
- AdvEvo-MARL: Shaping Internalized Safety through Adversarial Co-Evolution in Multi-Agent Reinforcement Learning (ICML 2026 · similarity 0.87)
- Adversaries With Incentives: A Strategic Alternative to Adversarial Robustness (ICLR 2025 · similarity 0.86)
- Automated Red Teaming with GOAT: the Generative Offensive Agent Tester (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
