Research map / Architectures and efficiency / Pruning and efficient tuning
Sparsity Winning Twice: Better Robust Generalization from More Efficient Training
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use early-ticket static sparsity for adversarial training | ResNet-18 · CIFAR-10 | robust generalization gap: 38.82 -> 23.37 | Helped |
| Use dynamic sparse connectivity during adversarial training | ResNet-18 · CIFAR-100 | robust generalization gap: 54.56 -> 27.46 | Helped |
| Adapt sparsity during dynamic sparse adversarial training | ResNet-18 · CIFAR-100 | robust generalization gap: 54.56 -> 20.12 | Helped |
| Replace dense AT with adaptive dynamic sparsity | ResNet-18 · CIFAR-100 | training FLOPs: 772.41 -> 100.90 | Helped |
| Use early sparse tickets found by standard SGD instead of Fast AT or PGD-10 (ablation) | Robust Bird · CIFAR-100, 90% sparsity | robust generalization gap: 18.37 vs 20.23 and 20.03 | Helped |
| Initialize Flying Bird+ with SNIP rather than the other tested initializers (ablation) | Flying Bird+ · CIFAR-10, ResNet-18, 80% initial sparsity | best robust accuracy: 51.30 vs 49.09, 50.57, 50.76, 50.56, 50.88 | Helped |
| Initialize Flying Bird+ with IGQ rather than SNIP (ablation) | Flying Bird+ · CIFAR-10, ResNet-18, 80% initial sparsity | robust generalization gap: 15.16 vs 15.15 | Helped |
| Update sparse connections every 100 rather than 2000 iterations (ablation) | Flying Bird+ · CIFAR-10, ResNet-18, 80% initial sparsity | robust generalization gap: 19.17 -> 13.36 | Helped |
| Update sparse connections every 2000 rather than 100 iterations (ablation) | Flying Bird+ · CIFAR-10, ResNet-18, 80% initial sparsity | best robust accuracy: 50.32 -> 51.19 | Helped |
| Use 500, 1000, 5000, or 10000 iteration update intervals instead of 2000 (ablation) | Flying Bird+ · CIFAR-10, ResNet-18, 80% initial sparsity | best robust accuracy: 51.19 -> 50.57; 50.99; 50.39; 50.08 | Hurt |
Closest papers by meaning
- Fantastic Robustness Measures: The Secrets of Robust Generalization (NeurIPS 2023 · similarity 0.91)
- Sparsity May Cry: Let Us Fail (Current) Sparse Neural Networks Together! (ICLR 2023 · similarity 0.90)
- Overfitting in adversarially robust deep learning (ICML 2020 · similarity 0.90)
- Adversarial Training Can Provably Improve Robustness: Theoretical Analysis of Feature Learning Process Under Structured Data (ICLR 2025 · similarity 0.89)
- Sparse Cocktail: Every Sparse Pattern Every Sparse Ratio All At Once (ICML 2024 · similarity 0.88)
- Adversarial generalization of unfolding (model-based) networks (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.