Research map / Adversarial robustness and security / Backdoor and poisoning

Adversarial Inception Backdoor Attacks against Reinforcement Learning

ICML 2025 · Adversarial robustness and security › Backdoor and poisoning · approach: Trigger-based backdoor attacks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes adversarial inception, a training-time backdoor attack on deep RL that manipulates stored trajectories (trigger insertion plus target-action replacement with bonus-compensated rewards) so attacks succeed while respecting benign reward bounds.

Problem

Existing DRL backdoor poisoning attacks rely on reward perturbations far outside the environment's natural reward bounds, making them detectable and ineffective once rewards are clipped or normalized.

Approach

Manipulate offline training trajectories: insert the trigger into prior observations and replace high-return actions with the target action, while assigning rewards only within the benign min/max range and using a DQN-derived advantage estimate to choose which steps to poison; theoretically this makes optimal policies of the adversarial MDP both attack-successful and stealthy under reward constrai

Key results

• Q-Incept achieves 100% ASR on Qbert, Frogger, Pacman, Breakout, and Safety Car at poisoning rates of 0.05%-0.3% under constrained rewards, versus at most 55.6% for SleeperNets and 22.5% for TrojDRL on Qbert.
• On CAGE-2 and Highway Merge, Q-Incept reaches 93.21% and 61.60% ASR at beta 1% and 10% respectively, where baselines stay below 6%.
• Without adversarial inception (Q-Vanilla) on CAGE-2, ASR falls below 5%, indicating inception is the key component.

Measured effects reported in the paper

Surprising findings

• Constrained-reward SleeperNets and TrojDRL drop to near-zero ASR on CAGE-2 and Highway Merge, showing prior attacks fail once reward bounds are enforced.
• Highway Merge's positive-only reward constraints limit the adversary from penalizing non-target actions, suggesting MDP construction can mitigate backdoors.

Limitations

• Q-Incept does not reach 100% ASR on CAGE-2 and Highway Merge, the hardest environments.
• Attack relies on an approximate Q-function (DQN) rather than the true optimal Q-function, so theoretical guarantees only hold for the idealized adversarial MDP.
• Evaluated only with PPO and a single trigger design; the paper's threat model assumes outer-loop access to training trajectories.

Experimental setup

Datasets: Atari Q*bert Atari Frogger Atari Pacman Atari Breakout CAGE Challenge 2 Highway Merge Safety Car

Benchmarks: Atari Q*bert Atari Frogger Atari Pacman Atari Breakout CAGE-2 Highway Merge Safety Car

Metrics: attack success rate (ASR) benign return (BR)

Baselines: SleeperNets TrojDRL

Builds on: TrojDRL SleeperNets DQN PPO

Closest papers by meaning

• Beyond Training-time Poisoning: Component-level and Post-training Backdoors in Deep Reinforcement Learning (AAAI 2026 · similarity 0.92)
• BadRL: Sparse Targeted Backdoor Attack against Reinforcement Learning (AAAI 2024 · similarity 0.89)
• SHINE: Shielding Backdoors in Deep Reinforcement Learning (ICML 2024 · similarity 0.87)
• PolicyGuard: Towards Test-time and Step-level Adversary Defense for Reinforcement Learning Agent (ICML 2026 · similarity 0.87)
• RAT: Adversarial Attacks on Deep Reinforcement Agents for Targeted Behaviors (AAAI 2025 · similarity 0.87)
• Optimal Attack and Defense for Reinforcement Learning (AAAI 2024 · similarity 0.86)