Research map / Adversarial robustness and security / Certified and empirical robustness
Et Tu Certifications: Robustness Certificates Yield Better Adversarial Examples
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes the Certification Aware Attack, which uses randomised-smoothing (and IBP) certificates to guide step sizes and refine norm-minimising adversarial examples, and argues that publishing certificates can reduce security.
Problem
Certified robustness guarantees (e.g., randomised smoothing) bound the region free of adversarial examples, but it is unclear whether releasing those certificates helps attackers find smaller, harder-to-detect adversarial examples against certified models.
Approach
Treat the certified radius at each iterate as a known lower bound on the distance to the decision boundary: step sizes are chosen to leave previously certified balls (to search) and, after an adversarial example is found, steps stay within certified radii of the new class to shrink the perturbation norm; the argmax is replaced by a Gumbel-softmax for differentiability.
Key results
- On CIFAR-10 at sigma=0.5 the attack is smallest for 87% of attacked samples (vs 4% for PGD), with %-C of 56 vs 72 for PGD.
- On MNIST at sigma=1.0 the attack achieves 97% best proportion with %-C 90 vs 123 for PGD at 99% success rate.
- Reported median certification radii are about 11% smaller for MNIST, 12% for CIFAR-10 and 52% smaller for ImageNet than the next best attack, and the attack is faster than baselines except DeepFool.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Certification Aware Attack instead of competing attacks | certified models · MNIST, CIFAR-10, and ImageNet | median attack size relative to Cohen certificate: 24% reduction in the median attack size relative to the next best attack | Helped |
| Use Certification Aware Attack instead of the next best technique | certified models · tested experiments | adversarial perturbation size: up to $55 \%$ decrease | Helped |
| Use Certification Aware Attack instead of PGD | randomised-smoothed certified models · MNIST, CIFAR-10, and ImageNet | median attack size r50 | Helped |
| Use Certification Aware Attack instead of PGD | IBP-certified model · MNIST | attack success rate | Helped |
| Increase smoothing noise σ | randomised-smoothing certified models · MNIST, CIFAR-10, and ImageNet | attack size relative to certified guarantee | Hurt |
| Train with MACER rather than the main-body training setup | CIFAR-10 ResNet-110 · CIFAR-10 | DeepFool attack success rate | Helped |
Surprising findings
- On ImageNet at sigma=1.0 the attack succeeds on only 52% of samples, lower than PGD's 91%, indicating the parameter range may need extension for large datasets.
- Releasing certificates can be exploited: certified radii can be reconstructed from class expectations and sigma, so the authors argue only class labels should be released.
Limitations
- Attack success rate on ImageNet is substantially lower than PGD (52% vs 91% at sigma=1.0).
- Assumes white-box access to sigma and the smoothed model; the paper's own sigma-estimation experiments are only partial.
- Restricting to a single shared class per attack and reliance on the argmax being replaced by Gumbel softmax for differentiability.
Experimental setup
Datasets: MNIST CIFAR-10 ImageNet
Benchmarks: MNIST CIFAR-10 ImageNet
Metrics: attack success rate best proportion median attack radius r50 percentage difference to certified radius (%-C) attack time
Baselines: PGD Carlini-Wagner AutoAttack DeepFool MACER
Builds on: Cohen et al. randomised smoothing PGD Carlini-Wagner AutoAttack DeepFool
Closest papers by meaning
- Position: Certified Robustness Does Not (Yet) Imply Model Security (ICML 2025 · similarity 0.92)
- Double Bubble, Toil and Trouble: Enhancing Certified Robustness through Transitivity (NeurIPS 2022 · similarity 0.92)
- On the Scalability of Certified Adversarial Robustness with Generated Data (NeurIPS 2024 · similarity 0.91)
- Learning Better Certified Models from Empirically-Robust Teachers (NeurIPS 2026 · similarity 0.90)
- Provably Cost-Sensitive Adversarial Defense via Randomized Smoothing (ICML 2025 · similarity 0.88)
- A Recipe for Improved Certifiable Robustness (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.