Research map / Adversarial robustness and security / Certified and empirical robustness
Layer-Aware Analysis of Catastrophic Overfitting: Revealing the Pseudo-Robust Shortcut Dependency
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Analyzes layer-wise changes during catastrophic overfitting in single-step adversarial training and proposes LAP, a layer-aware adversarial weight perturbation that suppresses pseudo-robust shortcuts and prevents CO at no extra cost.
Problem
Catastrophic overfitting (CO) in single-step adversarial training, where robustness collapses and decision boundaries become distorted, yielding models that resist FGSM-type attacks but fail under multi-step attacks; the layer-level causes remain unclear.
Approach
Layer-wise analysis shows former layers become most sensitive during CO due to large-weight 'pseudo-robust shortcuts'; LAP applies adaptive weight perturbations whose magnitude decreases from former to latter layers, generated jointly with input perturbations so training cost stays close to FGSM.
Key results
- On CIFAR-10 at 16/255, R-LAP reaches 15.69% AutoAttack accuracy versus 0.00% for R-FGSM, which suffers catastrophic overfitting.
- N-LAP achieves the best AutoAttack accuracy among single-step methods on CIFAR-10 at 8/255 (44.97%) and 32/255 (8.85%), with R-FGSM collapsing to 0% at 12/255 and above.
- LAP adds roughly 7% training time over FGSM (11.8s vs 11.0s per epoch on an RTX 4090), versus 59.1s for PGD-10.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add LAP to N-FGSM | N-FGSM · CIFAR-10, 16/255 | Auto Attack accuracy: 0.00{ $\pm$0.00} -> \textbf{19.55}{ $\pm$0.18} | Helped |
| Add LAP to N-FGSM | N-FGSM · CIFAR-10, 32/255 | Auto Attack accuracy: 0.00{ $\pm$0.00} -> 8.85{ $\pm$1.48} | Helped |
| Add LAP to N-FGSM | N-FGSM · CIFAR-100, 16/255 | Auto Attack accuracy: 10.22{ $\pm$0.18} -> \textbf{10.40}{ $\pm$0.14} | Helped |
| Add LAP to N-FGSM | N-FGSM · CIFAR-100, 32/255 | Auto Attack accuracy: 0.00{ $\pm$0.00} -> \textbf{3.45}{ $\pm$0.28} | Helped |
| Remove 10% of large weights in former layers of a CO-affected model (ablation) | CO-affected model · FGSM attack | accuracy: 22% reduction | Hurt |
| Remove 10% of large weights in former layers of a CO-affected model (ablation) | CO-affected model · PGD attack | accuracy: 2.65% | Helped |
| Use Original AWP instead of LAP (ablation) | R-LAP, CIFAR-10 16/255 · CIFAR-10, 16/255 | Auto Attack accuracy: 15.69{ $\pm$0.28} -> \,\,\,0.00{ $\pm$0.00} | Hurt |
| Use Modified AWP with accumulated perturbation but without layer-aware scaling (ablation) | R-LAP, CIFAR-10 16/255 · CIFAR-10, 16/255 | Auto Attack accuracy: 15.69{ $\pm$0.28} -> 12.53{ $\pm$0.98} | Hurt |
| Add an extra backward propagation (LAP-A) (ablation) | LAP · CIFAR-10, 16/255 | Auto Attack accuracy: 15.69{ $\pm$0.28} -> \textbf{15.72}{ $\pm$0.01} | No clear effect |
| Replace adversarial weight perturbations with random weight perturbations (LAP-R) (ablation) | LAP · CIFAR-10, 16/255 | Auto Attack accuracy: 15.69{ $\pm$0.28} -> 11.22{ $\pm$0.10} | Hurt |
| Use L-infinity norm instead of L2 norm for weight perturbations (ablation) | LAP · CIFAR-10, 16/255 | Auto Attack accuracy: 15.69{ $\pm$0.28} -> 13.67{ $\pm$0.35} | Hurt |
| Increase input perturbation size alpha (ablation) | LAP · CIFAR-10, 16/255 | Natural accuracy | Hurt |
Surprising findings
- Random removal of weights in former layers of a CO-affected model lowers FGSM accuracy while slightly raising PGD accuracy, a performance paradox.
- Original AWP fails to prevent CO (0.00% AutoAttack accuracy at 16/255 for LAP ablation), indicating that naive weight perturbation does not suffice.
Limitations
- Hyperparameter beta must be tuned per noise magnitude and per baseline (V/R/N-LAP), with separate settings for each architecture and dataset.
- Theoretical analysis is a PAC-Bayes bound derived under assumptions (Lipschitz loss, Gaussian prior), and the paper does not show the bound is tight.
- Robust accuracy under extreme noise (e.g., 32/255) remains low for LAP variants (e.g., 8.85% AutoAttack for N-LAP on CIFAR-10).
Experimental setup
Datasets: CIFAR-10 CIFAR-100 Tiny-ImageNet
Benchmarks: CIFAR-10 CIFAR-100 Tiny-ImageNet
Metrics: natural accuracy AutoAttack accuracy PGD-50-10 accuracy training time
Baselines: V-FGSM R-FGSM N-FGSM FreeAT Grad Align ZeroGrad MultiGrad PGD-2 PGD-10 AWP
Builds on: Adversarial Weight Perturbation (AWP) Sharpness-Aware Minimization (SAM) R-FGSM N-FGSM PGD adversarial training
Closest papers by meaning
- Contrasting Adversarial Perturbations: The Space of Harmless Perturbations (AAAI 2025 · similarity 0.88)
- Theoretical Analysis of Robust Overfitting for Wide DNNs: An NTK Approach (ICLR 2024 · similarity 0.88)
- Benign Overfitting in Adversarial Training of Neural Networks (ICML 2024 · similarity 0.88)
- The Effectiveness of Random Forgetting for Robust Generalization (ICLR 2024 · similarity 0.87)
- Expressive Losses for Verified Robustness via Convex Combinations (ICLR 2024 · similarity 0.87)
- Enhancing Adversarial Transferability with Checkpoints of a Single Model's Training (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.