Research map / Adversarial robustness and security / Attacks on text and graphs
RAT: Adversarial Attacks on Deep Reinforcement Agents for Targeted Behaviors
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RAT, a targeted behavior attack on DRL agents that learns a preference-based intention policy as a behavioral target and trains an adversary plus a weighting function over the replay buffer via bi-level optimization.
Problem
Evaluating robustness of deep RL agents against targeted behavior attacks that induce human-preferred specific behaviors, since reward-reduction attacks are too generic and prior targeted attacks need predefined target states or policies or victim training rewards.
Approach
Train an intention policy from human preference feedback (PbRL with SAC) as the behavioral target, train a state-perturbation adversary to make the victim's perturbed policy match it via a KL objective, and learn a weighting function that re-weights the replay buffer's state occupancy through a bi-level optimization with convergence analysis.
Key results
- On Meta-world manipulation and opposite-behavior scenarios, RAT reaches e.g. 87.00% on Door Lock and 97.00% on Door Unlock, substantially above SA-RL, PA-AD and Random baselines.
- Ablation: removing the intention policy drops Drawer Open success from 99.1% to 21.7% and Faucet Close from 95.1% to 13.0%.
- RAT-WocaR achieves average reward 2052 on one robust-training task versus 825 for ATLA-PPO, per the robust-agent table.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use RAT instead of the strongest listed baseline attack | SAC victim agents · Meta-World manipulation and opposite-behavior tasks | attack success rate: Door Lock: 76.50 ± 14.97 -> 87.00 ± 10.00; Door Unlock: 11.11 ± 13.43 -> 97.00 ± 6.63; Window Open: 30.00 ± 21.19 -> 72. | Mixed |
| Add the intention policy πθ (ablation) | RAT · Drawer Open, Drawer Close, Faucet Open, Faucet Close | attack success rate: Drawer Open: 21.7% -> 99.1%; Drawer Close: 8.0% -> 80.9%; Faucet Open: 0.0% -> 84.4%; Faucet Close: 13.0% -> 95.1% | Helped |
| Add the weighting function hω (ablation) | RAT · Drawer Open, Drawer Close, Faucet Open, Faucet Close | attack success rate: Drawer Open: 91.3% -> 99.1%; Drawer Close: 70.2% -> 80.9%; Faucet Open: 89.8% -> 84.4%; Faucet Close: 94.1% -> 95.1% | Mixed |
| Add the combined behavior policy (ablation) | RAT · Drawer Open, Drawer Close, Faucet Open, Faucet Close | attack success rate: 68.0% -> 99.1%; 26.0% -> 80.9%; 57.0% -> 84.4%; 59.1% -> 95.1% | Helped |
| Train Decision Transformer victims with RAT-guided adversarial perturbations | Decision Transformer · MuJoCo Cheetah and Walker | targeted behavior success | Helped |
| Use RAT-ATLA or RAT-WocaR adversarial training | SAC agents · Meta-World manipulation tasks | episode reward under attack | Helped |
Surprising findings
- Oracle-reward SA-RL and PA-AD perform poorly on several opposite-behavior tasks, which the authors attribute to distribution drift.
- Removing the weighting function slightly improves Faucet Open success (89.8% vs 84.4%).
Limitations
- Relies on a scripted teacher rather than real human preference labels.
- Convergence guarantees rely on assumptions (Lipschitz smoothness, bounded gradients) and a one-step approximation of the inner argmin.
Experimental setup
Datasets: Meta-world manipulation tasks MuJoCo locomotion tasks (D4RL)
Benchmarks: Meta-world MuJoCo
Metrics: attack success rate average episode reward
Baselines: Random SA-RL PA-AD ATLA-PPO PA-ATLA WocaR-RL
Builds on: PEBBLE Soft Actor-Critic SA-RL PA-AD ATLA WocaR-RL Decision Transformer
Closest papers by meaning
- Adversarial Inception Backdoor Attacks against Reinforcement Learning (ICML 2025 · similarity 0.87)
- AdvEvo-MARL: Shaping Internalized Safety through Adversarial Co-Evolution in Multi-Agent Reinforcement Learning (ICML 2026 · similarity 0.87)
- Rethinking Adversarial Policies: A Generalized Attack Formulation and Provable Defense in RL (ICLR 2024 · similarity 0.86)
- Optimal Attack and Defense for Reinforcement Learning (AAAI 2024 · similarity 0.86)
- Representation-Based Robustness in Goal-Conditioned Reinforcement Learning (AAAI 2024 · similarity 0.85)
- PolicyGuard: Towards Test-time and Step-level Adversary Defense for Reinforcement Learning Agent (ICML 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.