Research map / Adversarial robustness and security / Certified and empirical robustness
Understanding Robust Overfitting of Adversarial Training and Beyond
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Remove small-loss data (data ablation) from training from the start (ablation) | AT on CIFAR10 PreAct ResNet-18 L_inf eps=8 · CIFAR10 | robust overfitting gap (PGD-20) | Helped |
| Remove large-loss data before robust overfitting occurs (ablation) | AT on CIFAR10 PreAct ResNet-18 L_inf eps=8 · CIFAR10 | robust overfitting gap (PGD-20) | No clear effect |
| Add MLCAT weight perturbation on small-loss data | AT on CIFAR10 PreAct ResNet-18 L_inf · CIFAR10 PGD-20 | Best-Last gap (pp): -7.86 to -0.83 | Helped |
| Add MLCAT loss scaling on small-loss data | AT on CIFAR10 PreAct ResNet-18 L_inf · CIFAR10 PGD-20 | Best-Last gap (pp): -7.86 to -0.03 | Helped |
| Add MLCAT loss scaling on small-loss data | AT on CIFAR10 AA L_inf · CIFAR10 AA | Last robust acc (%): 26.93 vs 42.08 | Hurt |
| Add MLCAT weight perturbation on small-loss data | AT on CIFAR10 AA L_inf PreAct ResNet-18 · CIFAR10 AA | Last robust acc (%): 50.32 vs 42.08 | Helped |
| Add MLCAT weight perturbation on small-loss data | AT on CIFAR100 PreAct ResNet-18 L_inf · CIFAR100 PGD-20 | Best-Last gap (pp): -7.62 to -0.70 | Helped |
| Add MLCAT loss scaling on small-loss data | AT on CIFAR100 PreAct ResNet-18 L_inf · CIFAR100 PGD-20 | Best robust acc (%): 20.09 vs 28.01 | Hurt |
| Increase minimum loss threshold l_min from small to ~3 (ablation) | MLCAT_LS / MLCAT_WP on CIFAR10 PreAct ResNet-18 · CIFAR10 PGD-20 | robust gap and robust accuracy | Mixed |
| Decrease (subtractive) instead of increase loss of small-loss data (ablation) | MLCAT_LS / MLCAT_WP · CIFAR10 | robust overfitting and robustness | Hurt |
| Extend MLCAT weight perturbation to TRADES | TRADES on CIFAR10 PreAct ResNet-18 L_inf · CIFAR10 PGD-20 | Best-Last gap (pp): -3.53 to -0.29 | Helped |
| Use MLCAT weight perturbation instead of AWP | AWP on CIFAR10 PreAct ResNet-18 L_inf · CIFAR10 AA | Best robust acc (%): 50.70 vs 49.94 | Helped |
Closest papers by meaning
- Overfitting in adversarially robust deep learning (ICML 2020 · similarity 0.92)
- Revisiting Adversarial Training Under Long-Tailed Distributions (CVPR 2024 · similarity 0.90)
- Expressive Losses for Verified Robustness via Convex Combinations (ICLR 2024 · similarity 0.89)
- Consistency Regularization for Adversarial Robustness (AAAI 2022 · similarity 0.89)
- Adversarial Training Can Provably Improve Robustness: Theoretical Analysis of Feature Learning Process Under Structured Data (ICLR 2025 · similarity 0.89)
- Layer-Aware Analysis of Catastrophic Overfitting: Revealing the Pseudo-Robust Shortcut Dependency (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.