Research map / Adversarial robustness and security / Attacks on text and graphs
Embodied Active Defense: Leveraging Recurrent Feedback to Counter Adversarial Patches
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Embodied Active Defense (EAD), a recurrent perception-plus-policy agent that actively moves its camera to gather further observations and thereby counters adversarial patches on face recognition and object detection in 3D scenes.
Problem
Adversarial patches in 3D physical environments fool DNNs from a single passive viewpoint, and existing defenses rely on single observations or prior adversary knowledge, failing against unseen or adaptive attacks.
Approach
An embodied agent with a recurrent perception model and a policy model that chooses camera rotations to collect informative observations, trained end-to-end through a differentiable deterministic environment approximation and attack-agnostic uniformly sampled surrogate patches (USAP), with a theoretical link to greedy informative exploration.
Key results
- On face recognition, EAD reaches 90.45% standard accuracy (undefended 88.86%) and reduces 3DAdv impersonation ASR to 7.2% vs 89.0% undefended.
- Averaged across unseen attacks, EAD reduces attack success rate by about 95% according to the abstract.
- On CARLA object detection, EAD achieves 39.4% mAP under MIM versus 30.6% for the undefended Mask-RCNN, with clean mAP preserved at 46.6%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use EAD instead of the undefended model | IResNet-50 ArcFace · CelebA-3D standard evaluation | accuracy (%): 88.86 -> 90.45 | Helped |
| Use EAD instead of PZ† | IResNet-50 ArcFace · CelebA-3D dodging (3DAdv) | ASR (%): 20.4 -> 13.7 | Helped |
| Use EAD instead of PZ† | IResNet-50 ArcFace · CelebA-3D impersonation (3DAdv) | ASR (%): 27.4 -> 7.2 | Helped |
| Use EAD instead of the strongest listed defense baseline | Mask-RCNN · CARLA-GeAR (MIM) | mAP (%): 36.5 -> 39.4 | Helped |
| Use EAD instead of the strongest listed defense baseline | Mask-RCNN · CARLA-GeAR (TIM) | mAP (%): 37.8 -> 39.3 | Helped |
| Use EAD instead of the strongest listed defense baseline | Mask-RCNN · CARLA-GeAR (SIB) | mAP (%): 36.4 -> 39.5 | Helped |
| Add the learned policy model to the perception model (ablation) | EAD · CelebA-3D impersonation (3DAdv) | ASR (%): 30.77 ± 1.81 -> 11.34 | Helped |
| Add USAP patch training to the perception and policy models (ablation) | EAD · CelebA-3D impersonation (adaptive) | ASR (%): 15.63 -> 8.33 | Helped |
| Use random movement instead of the learned policy (ablation) | EAD · CelebA-3D impersonation (3DAdv) | ASR (%): 11.34 -> 76.77 ± 3.34 | Hurt |
Surprising findings
- EAD's clean accuracy slightly exceeds the undefended model's accuracy (90.45% vs 88.86%).
- A multi-view ensemble with random movement fails to counter 3DAdv (76.77% ASR), indicating EAD's gains are not merely from viewpoint changes.
Limitations
- Requires additional hardware or computation for dynamic interaction and temporal processing, as stated in the ethics statement.
- Evaluation is limited to simulated environments (EG3D rendering and CARLA) and two tasks.
- Adaptive attacks against the full pipeline are costly, requiring gradient checkpointing and approximations via a uniform superset policy.
Experimental setup
Datasets: CelebA CelebA-3D LFW MS1MV3 COCO CARLA-GeAR
Benchmarks: CelebA-3D (face recognition, LFW protocol) CARLA-GeAR object detection scenes
Metrics: standard accuracy attack success rate (ASR) mAP
Baselines: Undefended JPEG LGS SAC PatchZero DOA Random Movement
Builds on: Decision Transformer EG3D Face3DAdv adversarial training InfoNCE
Closest papers by meaning
- A Unified, Resilient, and Explainable Adversarial Patch Detector (CVPR 2025 · similarity 0.86)
- AdvEDM: Fine-grained Adversarial Attack against VLM-based Embodied Agents (NeurIPS 2025 · similarity 0.85)
- Adversarial Robust Safeguard for Evading Deep Facial Manipulation (AAAI 2024 · similarity 0.85)
- AdvAD: Exploring Non-Parametric Diffusion for Imperceptible Adversarial Attacks (NeurIPS 2024 · similarity 0.85)
- PAD: Patch-Agnostic Defense against Adversarial Patch Attacks (CVPR 2024 · similarity 0.84)
- MAGIC: Mastering Physical Adversarial Generation in Context Through Collaborative LLM Agents (AAAI 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.