Research map / Adversarial robustness and security / Certified and empirical robustness
On the Scalability of Certified Adversarial Robustness with Generated Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that adding EDM-generated CIFAR-10/100 images to training substantially improves deterministic Lipschitz-based certified defenses, reaching new state-of-the-art certified accuracy on CIFAR-10 for both l_inf and l_2 threat models.
Problem
Certified robustness of deterministic certified defenses remains low, and it is unclear whether the generated-data scaling recipe that helps empirical adversarial training also helps certified training.
Approach
Transfer the generated-data recipe (EDM-generated images mixed 70/30 with real data per batch) from adversarial training to deterministic certified training of Lipschitz-bounded networks, and systematically study how data amount, model size, epochs, and regularization affect scaling, contrasting the results with empirical robustness.
Key results
- SortNet without dropout reaches 41.78% certified accuracy on CIFAR-10 (l_inf, 8/255) with 10M generated images, +1.39 points over prior best.
- LOT-L reaches 69.05% certified accuracy on CIFAR-10 (l2, 36/255) with 5M generated images, +3.95 points over prior best.
- Certified accuracy gains saturate beyond about 1M generated images, unlike empirical adversarial training which keeps improving with more data.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train with auxiliary EDM-generated data | SortNet without dropout · CIFAR-10 (ℓ∞, ϵ = 8/255) | certified accuracy: 36.50 -> 41.78 | Helped |
| Train with auxiliary EDM-generated data | LOT-L · CIFAR-10 (ℓ2, ϵ = 36/255) | certified accuracy: 64.39 -> 69.05 | Helped |
| Train with auxiliary EDM-generated data | SortNet without dropout · CIFAR-100 (ℓ∞, ϵ = 8/255) | certified accuracy: 9.20 -> 17.28 | Helped |
| Train with auxiliary EDM-generated data | LOT-L · CIFAR-100 (ℓ2, ϵ = 36/255) | certified accuracy: 32.92 -> 36.56 | Helped |
| Remove dropout (ablation) | SortNet with auxiliary data · CIFAR-10 (ℓ∞, ϵ = 8/255) | certified accuracy: 41.32 -> 41.78 | Helped |
| Remove dropout (ablation) | SortNet without auxiliary data · CIFAR-10 (ℓ∞, ϵ = 8/255) | certified accuracy: 39.72 -> 37.44 | Hurt |
| Increase training epochs (ablation) | certified models trained with auxiliary data · CIFAR-10 | certified accuracy | Helped |
| Use cosine annealing instead of a multi-step scheduler (ablation) | LOT with auxiliary data · CIFAR-10 (ℓ2, ϵ = 36/255) | certified accuracy: +1.12 points | Helped |
| Increase generated-data amount beyond 1m (ablation) | Lipschitz-bound certified models · CIFAR-10 | certified accuracy | No clear effect |
| Use only generated images instead of only real images (ablation) | ℓ∞-dist Net and LOT · CIFAR-10 | accuracy | Helped |
| Add generated data to convex bound propagation training (ablation) | MTL-IBP · CIFAR-10 (ℓ∞, ϵ = 8/255) | clean accuracy: 35.08% -> 0.00% | Hurt |
| Scale auxiliary generated data from 1m to 100m (ablation) | adversarial training · CIFAR-10 | robust accuracy: +2.65 and +2.52% | Helped |
Surprising findings
- Scaling generated data beyond 1M CIFAR-10 images does not further improve certified accuracy, contrary to adversarial training scaling reported by Gowal et al.
- Convex bound propagation (MTL-IBP) showed no benefit from generated data and sometimes worse results.
Limitations
- Focused mainly on deterministic Lipschitz-bound methods; probabilistic and other deterministic approaches largely unexplored.
- Single run per configuration due to compute cost, so no error bars; small hyperparameter changes shift results by 0.5-1 points.
- No experiments on ImageNet, since only one evaluated model supports it.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 EDM-generated CIFAR-10 images (from Wang et al.)
Benchmarks: CIFAR-10 CIFAR-100
Metrics: clean accuracy certified accuracy
Baselines: SortNet ℓ∞-dist Net LOT GloroNet MTL-IBP Hu et al. GloroNet DDPM
Builds on: Gowal et al. generated data for adversarial training Wang et al. DM-based adversarial training elucidating diffusion models SortNet LOT GloroNet
Closest papers by meaning
- Certified Adversarial Robustness with Additive Noise (NeurIPS 2019 · similarity 0.94)
- Et Tu Certifications: Robustness Certificates Yield Better Adversarial Examples (ICML 2024 · similarity 0.91)
- Position: Certified Robustness Does Not (Yet) Imply Model Security (ICML 2025 · similarity 0.90)
- Learning Better Certified Models from Empirically-Robust Teachers (NeurIPS 2026 · similarity 0.90)
- Provably Cost-Sensitive Adversarial Defense via Randomized Smoothing (ICML 2025 · similarity 0.90)
- Certified Causal Defense with Generalizable Robustness (AAAI 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.