Research map / Adversarial robustness and security / Attacks on text and graphs
Detecting Adversarial Data Using Perturbation Forgery
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Perturbation Forgery, which trains a binary adversarial detector on pseudo-adversarial data made by perturbing a Gaussian noise distribution (from FGSM) and sparsifying it with saliency/GradCAM/Sobel masks, to generalize to unseen gradient, generative, and physical attacks.
Problem
Adversarial detectors trained for specific attacks generalize poorly to unseen gradient-based attacks and especially to generative-based attacks with imbalanced, anisotropic noise, and existing generalizing detectors add large inference overhead.
Approach
Model adversarial noise as Gaussian distributions, argue that perturbed variants of an attack's noise distribution form an open covering of all adversarial noise distributions, and synthesize localized pseudo-adversarial noise by sampling from perturbed distributions and masking it with saliency/GradCAM/Sobel sparse masks, so a detector trained on this data detects unseen attacks at low inference
Key results
- On ImageNet100 under eps=4/255, the method reaches AUROC 0.9931 on MIM and 0.9944 on MM, comparable to EPSAD's 0.9918 and 0.9923 but with far lower inference time (4.85 vs 396.81 s per 100 samples).
- On generative attacks on ImageNet100 it attains AUROC 0.9878 on CDA, 0.9678 on TTP and 0.9223 on Diff-Attack and Diff-PGD, outperforming EPSAD and SPAD on most of them.
- Ablation shows removing the Sobel sparsification drops TTP to 0.7235 and M3D to 0.6984 AUROC, while white-box detection averages about 0.91 AUROC on CIFAR-10.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add saliency detection model fs to the sparse mask (ablation) | Perturbation Forgery · ImageNet100 generative-based attacks | AUROC: w/o fs: 0.9755, 0.9585, 0.9320, 0.9197, 0.9135 | Helped |
| Add GradCAM model fc to the sparse mask (ablation) | Perturbation Forgery · ImageNet100 generative-based attacks | AUROC: w/o fc: 0.9864, 0.9510, 0.9262, 0.9031, 0.9062 | Helped |
| Add Sobel operation to sparsify the mask (ablation) | Perturbation Forgery · ImageNet100 generative-based attacks | AUROC: w/o Sobel: 0.9868, 0.7235, 0.6984, 0.5231, 0.5547 | Helped |
| Use 100, 250, 500, 750, 1000, or 2000 samples for distribution estimation (ablation) | Perturbation Forgery on CIFAR-10 · CIFAR-10, PGD | AUROC: 0.8278 0.9368 0.9866 0.9921 0.9965 0.9948 | Mixed |
| Train with Perturbation Forgery under a white-box attack setting | Perturbation Forgery on CIFAR-10 · CIFAR-10, white-box attacks | AUROC: 0.9180, 0.9168, 0.9062, 0.9031, 0.9019 | Mixed |
| Train with Perturbation Forgery rather than specialized detector baselines | ImageNet100 gradient-based attack detection · ImageNet100, MIM | AUROC: 0.9918 -> 0.9931 | Helped |
| Train with Perturbation Forgery rather than specialized detector baselines | ImageNet100 generative-based attack detection · ImageNet100, Diff-PGD | AUROC: 0.9143 -> 0.9223 | Helped |
Surprising findings
- EPSAD, a strong gradient-attack detector, drops to 0.1507 AUROC on Diff-PGD and 0.4700 on Diff-Attack, showing diffusion-based attacks evade it.
Limitations
- Relies on Gaussian modeling of adversarial noise, which is an assumption; the theory is heuristic.
- Performance drops when the distribution is estimated from fewer than 500 samples.
- Some generative attacks (e.g., Diff-Attack, Diff-PGD) reach only about 0.92 AUROC, lower than gradient-attack results.
Experimental setup
Datasets: ImageNet100 CIFAR-10 Makeup CelebA-HQ LFW
Benchmarks: ImageNet100 CIFAR-10 Makeup CelebA-HQ LFW
Metrics: AUROC inference time
Baselines: LID LiBRe SPAD EPSAD CNN-Detection LGrad Universal-Detector DIRE ODIN Luo et al. He et al.
Builds on: FGSM Torchattacks XceptionNet
Closest papers by meaning
- On Detecting Adversarial Perturbations (ICLR 2017 · similarity 0.89)
- Detecting Adversarial Data by Probing Multiple Perturbations Using Expected Perturbation Score (ICML 2023 · similarity 0.89)
- Mind the Gap: Detecting Black-box Adversarial Attacks in the Making through Query Update Analysis (CVPR 2025 · similarity 0.88)
- One Stone, Two Birds: Enhancing Adversarial Defense Through the Lens of Distributional Discrepancy (ICML 2025 · similarity 0.87)
- Wide Two-Layer Networks can Learn from Adversarial Perturbations (NeurIPS 2024 · similarity 0.86)
- Theoretical Understanding of Learning from Adversarial Perturbations (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
