Research map / Adversarial robustness and security / Black-box and transfer attacks
CosPGD: an efficient white-box adversarial attack for pixel-wise prediction tasks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CosPGD, a white-box adversarial attack for pixel-wise prediction tasks that scales each pixel's loss by the cosine alignment between prediction and target, outperforming PGD and SegPGD on segmentation, optical flow and image restoration.
Problem
Adversarial attacks for pixel-wise prediction tasks (semantic segmentation, optical flow, disparity, image restoration) are weak when they use classification-style PGD, and the prior pixel-wise attack SegPGD relies on discrete argmax-based loss scaling that is unstable and does not extend to regress
Approach
Scale the per-pixel loss by a continuous cosine-similarity alignment between the softmax-normalized prediction and the target (or one minus it for targeted attacks), giving a smooth, fully differentiable, task-agnostic PGD variant that balances the attack spatially.
Key results
- On PASCAL VOC 2012 with DeepLabV3, PGD reduces mIoU to 6.79% while SegPGD reaches 2.69%, and CosPGD brings it to almost 0% after 100 iterations.
- On ADE20K with SegFormer (MIT-B0), at epsilon 4/255 and 10 iterations, CosPGD reaches mIoU 0.09% versus 1.66% for PGD and 0.89% for SegPGD.
- For targeted optical flow attacks on RAFT, CosPGD reduces Sintel (final) epe to the target to as low as 1.55, and it outperforms PGD across KITTI-2015 and Sintel.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CosPGD instead of PGD | SegFormer (MIT-B0) · ADE20K, ε=2/255, 3 iterations | mIoU (%): 8.45 -> 5.37 | Helped |
| Use CosPGD instead of SegPGD | SegFormer (MIT-B0) · ADE20K, ε=2/255, 3 iterations | mIoU (%): 5.80 -> 5.37 | Helped |
| Use CosPGD instead of PGD | SegFormer (MIT-B0) · ADE20K, ε=4/255, 3 iterations | mIoU (%): 5.11 -> 1.66 | Helped |
| Use CosPGD instead of SegPGD | SegFormer (MIT-B0) · ADE20K, ε=4/255, 3 iterations | mIoU (%): 3.29 -> 1.66 | Helped |
| Increase the perturbation bound from ε=2/255 to ε=4/255 for CosPGD (ablation) | SegFormer (MIT-B0) · ADE20K, 3 iterations | mIoU (%): 5.37 -> 1.66 | Helped |
| Use CosPGD instead of PGD | DeepLabV3 · PASCAL VOC 2012, 100 iterations | mIoU (%): 6.79% -> almost 0% | Helped |
| Use CosPGD instead of SegPGD | DeepLabV3 · PASCAL VOC 2012, 100 iterations | mIoU (%): 2.69% -> almost 0% | Helped |
Surprising findings
- SegPGD at low iteration counts suggests PSPNet is more robust than DeepLabV3, while CosPGD correctly indicates the reverse even at few iterations.
- Under the reported settings, the per-iteration compute time of CosPGD is comparable to PGD and SegPGD.
Limitations
- Non-targeted CosPGD requires ground-truth labels, which many test sets do not provide.
- SegPGD-style binary weighting is not applicable to regression, so comparisons on optical flow are limited to PGD and PCFA.
- Evaluation of the attack is restricted to l-inf and l2 bounded perturbations with a fixed step size of 0.01 in the main experiments.
Experimental setup
Datasets: PASCAL VOC 2012 Cityscapes ADE20K KITTI 2015 MPI Sintel GoPro SSID
Benchmarks: PASCAL VOC 2012 KITTI 2015 MPI Sintel GoPro ADE20K Cityscapes SSID
Metrics: mIoU mAcc epe epe-f1-all PSNR SSIM
Baselines: PGD SegPGD PCFA
Builds on: PGD SegPGD FGSM PCFA
Closest papers by meaning
- RP-PGD: Boosting Segmentation Robustness with a Region-and-Prototype Based Adversarial Attack (AAAI 2025 · similarity 0.87)
- BadPart: Unified Black-box Adversarial Patch Attacks against Pixel-wise Regression Tasks (ICML 2024 · similarity 0.85)
- Vanish into Thin Air: Cross-prompt Universal Adversarial Attacks for SAM2 (NeurIPS 2025 · similarity 0.85)
- DeSparsify: Adversarial Attack Against Token Sparsification Mechanisms (NeurIPS 2024 · similarity 0.85)
- A Frustratingly Simple Yet Highly Effective Attack Baseline: Over 90% Success Rate Against the Strong Black-box Models of GPT-4.5/4o/o1 (NeurIPS 2025 · similarity 0.84)
- Adversarial Supervision Makes Layout-to-Image Diffusion Models Thrive (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.