Research map / Adversarial robustness and security / Attacks on text and graphs
Illusory Attacks: Information-theoretic detectability matters in adversarial attacks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces epsilon-illusory attacks, observation-space attacks on RL agents constrained by a trajectory-level KL detectability bound, learned via a dual ascent algorithm, showing they evade automated OOD detectors and human inspection while remaining effective.
Problem
Existing observation-space adversarial attacks on RL agents are effective but detectable by anomaly detectors and humans, and robustification becomes ineffective for large perturbation budgets.
Approach
Constrains the adversary so the victim's trajectory distribution under attack stays within a KL-divergence threshold epsilon of the unattacked distribution (information-theoretic detectability), and learns such attacks end-to-end via a dual ascent algorithm with a single-sample Monte Carlo KL estimator.
Key results
- Epsilon-illusory attacks achieve detection rates near 0-4% against a state-of-the-art OOD detector, versus about 76-100% for SA-MDP and MNP attacks, while still reducing victim reward (e.g., Pendulum B=0.2: -980 vs -1387 for SA-MDP at detection 3.6% vs 100%).
- In a study with n=10 participants, P(false | SA-MDP) was 81.4% and P(false | MNP) 83.3%, versus 32.4% for epsilon-illusory attacks and 34.2% for unattacked sequences; the latter difference could not be rejected.
- Randomized smoothing gave ~61% average improvement at B=0.05 but only ~15% at B=0.2 under attack, indicating robustification weakens for large budgets.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Constrain the attack to be ε-illusory instead of using SA-MDP | Adversarial attacks · Pendulum, B=0.2 | Detection Rate: 100$\pm$0.03 -> 3.6$\pm$0.02 | Helped |
| Constrain the attack to be ε-illusory instead of using SA-MDP | Adversarial attacks · HalfCheetah, B=0.2 | Victim reward: -1643.8$\pm$344.8 -> -240.6$\pm$ 18.0 | Helped |
| Use ε-illusory attack instead of SA-MDP | Adversarial attacks · CartPole, B=0.2 | Victim reward: 9.3$\pm$0.1 -> 11.0$\pm$0.5 | Hurt |
| Use perfect illusory attack instead of the unattacked environment | CartPole · CartPole | Victim reward: 500.0 -> 30.1$\pm$2.2 | Hurt |
| Use perfect illusory attack rather than SA-MDP | Adversarial attacks · Pendulum | Detection Rate: 100$\pm$0.03 -> 3.0$\pm$0.02 | Helped |
| Raise the attack budget from B=0.05 to B=0.2 | Randomized smoothing · RL environments | Performance improvement: 61% -> 15% | Hurt |
| Raise the attack budget from B=0.05 to B=0.2 | Adversarial pretraining (ATLA) · RL environments | Performance improvement: 10% -> 8% | Hurt |
| Use ε-illusory attacks instead of SA-MDP or MNP attacks | Attacks shown to human participants · Human study, both environments | P(false | attack): 81.4 ± 27.2 -> 32.4 ± 10.8 | Helped |
| Use ε-illusory attacks rather than unattacked sequences | Attacks shown to human participants · Human study, both environments | P(false | sequence): 34.2 ± 11.4 vs. 32.4 ± 10.8 | No clear effect |
| Increase ε | ε-illusory attack · Single-step stochastic MDP | Adversary return | Helped |
Surprising findings
- Humans classify epsilon-illusory attacked sequences as suspicious at a rate statistically indistinguishable from unattacked sequences.
- Perfect illusory attacks with a symmetric initial-state trick achieve nonzero victim degradation with exactly zero KL divergence.
Limitations
- Experiments confined to simulated low-dimensional control benchmarks; real-world domains not tested.
- Victim assumed to have contingency options upon detection; adversary assumed to know the state-transition function.
- Human study small (n=10) and uses short video sequences.
Experimental setup
Datasets: CartPole Pendulum Hopper HalfCheetah single-step stochastic MDP
Benchmarks: CartPole Pendulum Hopper HalfCheetah
Metrics: adversary score detection rate victim reward human detection probability
Baselines: SA-MDP attacks MNP attacks randomized smoothing ATLA
Builds on: SA-MDP RCPO PPO SAC CUSUM
Closest papers by meaning
- Optimal Attack and Defense for Reinforcement Learning (AAAI 2024 · similarity 0.85)
- Adversaries With Incentives: A Strategic Alternative to Adversarial Robustness (ICLR 2025 · similarity 0.85)
- Diffusion Guided Adversarial State Perturbations in Reinforcement Learning (NeurIPS 2025 · similarity 0.84)
- Adversarial Inception Backdoor Attacks against Reinforcement Learning (ICML 2025 · similarity 0.84)
- Practical Adversarial Attacks on Stochastic Bandits via Fake Data Injection (NeurIPS 2026 · similarity 0.83)
- RAT: Adversarial Attacks on Deep Reinforcement Agents for Targeted Behaviors (AAAI 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.