Research map / Adversarial robustness and security / Black-box and transfer attacks
Robustness benchmarks and defenses
Benchmark adversarial robustness and defend through architecture, denoising and loss design.
89 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2017–2026) · an approach in Black-box and transfer attacks, keyword cluster “query · black box · box adversarial”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
20 papers in 2023–24 and 17 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 22.5% to 14.7%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2017 · Adversarial Machine Learning at Scale (ICLR 2017 · 368 citations)
- 2018 · Ensemble Adversarial Training: Attacks and Defenses (ICLR 2018 · 2,314 citations)
- 2019 · Feature Denoising for Improving Adversarial Robustness (CVPR 2019 · 902 citations)
- 2020 · Benchmarking Adversarial Robustness on Image Classification (CVPR 2020 · 255 citations)
- 2021 · RobustBench: a standardized adversarial robustness benchmark (NeurIPS 2021 · 114 citations)
- 2022 · Zero-Query Transfer Attacks on Context-Aware Object Detectors (CVPR 2022 · 20 citations)
- 2023 · CodeAttack: Code-Based Adversarial Attacks for Pre-trained Programming Language Models (AAAI 2023 · 59 citations)
- 2024 · CosPGD: an efficient white-box adversarial attack for pixel-wise prediction tasks (ICML 2024 · 13 citations)
- 2025 · Reinforcement Learning Platform for Adversarial Black-box Attacks with Custom Distortion Filters (AAAI 2025 · 2 citations)
- 2026 · STARE: Step-wise Temporal Alignment and Red-teaming Engine for Multi-modal Toxicity Attack (ICML 2026 · 0 citations)
Earliest papers here
- Adversarial Machine Learning at Scale (ICLR 2017 · 368 citations)
- Ensemble Adversarial Training: Attacks and Defenses (ICLR 2018 · 2,314 citations)
- Defense Against Adversarial Attacks Using High-Level Representation Guided Denoiser (CVPR 2018 · 945 citations)
What has been tried
79 papers here report 833 measured changes (a component added, removed or swapped, compared with a baseline): helped 66%, no clear effect 8%, hurt 21%, mixed 5%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Increase the perturbation bound from ε=2/255 to ε=4/255 for CosPGD | ADE20K, 3 iterations · mIoU (%) | 5.37 -> 1.66 | Helped | CosPGD: an efficient white-box adversarial attack for pixel-wise prediction tasks (ICML 2024 · 13 citations) |
| place Accurate Cells before Robust Cells (A-A-R) instead of R-A-A | CIFAR-10 · PGD20 | 52.51% -> 55.87% | Helped | Towards Accurate and Robust Architectures via Neural Architecture Search (CVPR 2024 · 11 citations) |
| Use random initialization instead of target-repeat initialization | First 20 harmful strings · successful targets | only twice | Hurt | Query-Based Adversarial Prompt Generation (NeurIPS 2024 · 6 citations) |
| skip style-reference selection and use random style images | UCF-101 targeted · fooling rate | 55% -> 29% | Hurt | LogoStyleFool: Vitiating Video Recognition Systems via Logo Style Transfer (AAAI 2024 · 4 citations) |
| use Brightness filter instead of Gaussian Noise | ImageNet · AVG.Q | 166 -> 266 | Hurt | Reinforcement Learning Platform for Adversarial Black-box Attacks with Custom Distortion Filters (AAAI 2025 · 2 citations) |
| use ResNet-9 instead of ResNet-18 as the shadow model | CIFAR-10 (d) · Compensation Share ratio | 217.3% -> 206.3% | Hurt | Adversarial Attacks on Data Attribution (ICLR 2025 · 1 citation) |
| add probabilistic sampling | Monodepth2, 300K queries · DEE | 38.41 -> 54.88 | Helped | BadPart: Unified Black-box Adversarial Patch Attacks against Pixel-wise Regression Tasks (ICML 2024 · 1 citation) |
| Apply ILA directly to the full intermediate representation | AdvBench, query-specific suffixes, Llama-2-7B-Chat · match rate | 54% -> 44% | Hurt | Improved Generation of Adversarial Examples Against Safety-aligned LLMs (NeurIPS 2024 · 1 citation) |
| Use Uniform initialization instead of Range Prior | ImageNet ILSVRC2012 val, AlexNet · fooling ratio (%) | 91.60 -> 92.58 | Helped | Data-Free Universal Attack by Exploiting the Intrinsic Vulnerability of Deep Models (AAAI 2025 · 0 citations) |
| Use CEAttack++ settings, including a delete-word ranking scheme and larger synonym search | SST-2 · ASR | 14.58 -> 28.42 | Helped | Confidence Elicitation: A New Attack Vector for Large Language Models (ICLR 2025 · 0 citations) |
| randomly select source models instead of using the default selection | CIFAR-10 transfer attacks (TAPM) · adversarial accuracy | 88.6 -> 62.9 | Hurt | PubDef: Defending Against Transfer Attacks From Public Models (ICLR 2024 · 0 citations) |
| Increase UV wavelength | LFW, dodging attacks · ASR | Hurt | Omni-Angle Assault: An Invisible and Powerful Physical Adversarial Attack on Face Recognition (ICML 2025) |
What reviewers will expect
Datasets and baselines named in the 11 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- CIFAR-10 · 4 of 11 papers
- CIFAR-100 · 2 of 11 papers
- ImageNet-1k · 2 of 11 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 37 papers; from OpenAlex and the papers' affiliations).
- Tsinghua University · 2 papers
- Hewlett Packard Enterprise · 2 papers
- Sichuan University · 2 papers
Use it
Download BibTeX of 40 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Query-efficient black-box attacks (78 papers)
- Data-free substitute training (49 papers)
- Decision-based boundary attacks (24 papers)
Elsewhere in the topic
- Transferable adversarial examples (86 papers)
- Transfer attacks on face recognition (48 papers)
- Transfer attacks on vision transformers (9 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).