Research map / Adversarial robustness and security / Certified and empirical robustness
The Effectiveness of Random Forgetting for Robust Generalization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FOMO, an adversarial training paradigm that alternates random weight reinitialization of later layers (forgetting) with relearning and EMA-based consolidation into a stable model with KL consistency regularization, to reduce robust overfitting.
Problem
Robust overfitting in adversarial training, where robust test accuracy degrades with continued training, so the best-epoch and last-epoch robust accuracy gap is large and early stopping is needed.
Approach
Inspired by active forgetting in the brain, periodically randomly reinitialize a sparse subset of weights in the later layers during AT, then relearn adversarially, consolidating knowledge into an EMA stable model that is used for inference and regularizes training via KL consistency.
Key results
- On CIFAR-10 with PreActResNet-18, FOMO reaches 56.46% last-epoch PGD-20 robust accuracy versus 44.44% for PGD-AT, with best-last gap of -0.22 versus -7.88.
- Under AutoAttack on CIFAR-10 with PreActResNet-18, FOMO achieves 51.37% best and 51.28% last robust accuracy, versus 47.72% and 42.60% for PGD-AT.
- On CIFAR-100 under PGD-20, FOMO achieves 31.67% last-epoch robust accuracy versus 20.82% for PGD-AT.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FOMO instead of PGD-AT | PreAct-ResNet-18 adversarial training · CIFAR-10, PGD-20 last | robust accuracy: 44.44 -> 56.46 | Helped |
| Use FOMO instead of PGD-AT | WideResNet-34-10 adversarial training · CIFAR-10, PGD-20 last | robust accuracy: 48.16 -> 59.23 | Helped |
| Use FOMO instead of PGD-AT | PreAct-ResNet-18 adversarial training · CIFAR-100, PGD-20 last | robust accuracy: 20.82 -> 31.67 | Helped |
| Use FOMO instead of PGD-AT | PreAct-ResNet-18 adversarial training · SVHN, PGD-20 last | robust accuracy: 45.13 -> 59.06 | Helped |
| Use FOMO instead of MLCAT_WP | PreAct-ResNet-18 adversarial training · CIFAR-10, AutoAttack last | robust accuracy: 50.32 -> 51.28 | Helped |
| Use FOMO instead of MLCAT_WP | PreAct-ResNet-18 adversarial training · CIFAR-100, AutoAttack last | robust accuracy: 25.18 -> 27.49 | Helped |
| Add periodic forgetting of later-layer weights (ablation) | PGD-AT, PreAct-ResNet-18 · CIFAR-10, PGD-20 last | robust accuracy: 44.44 -> 48.86 | Helped |
| Add periodic forgetting of later-layer weights (ablation) | PGD-AT, PreAct-ResNet-18 · CIFAR-10, AutoAttack last | robust accuracy: 42.60 -> 44.63 | Helped |
| Add stable-model consolidation (ablation) | PGD-AT + Forgetting · CIFAR-10, PGD-20 last | robust accuracy: 48.86 -> 52.80 | Helped |
| Add stable-model consolidation (ablation) | PGD-AT + Forgetting · CIFAR-10, AutoAttack last | robust accuracy: 44.63 -> 49.30 | Helped |
| Add consistency regularization from the stable model (ablation) | PGD-AT + Forgetting + Consolidation · CIFAR-10, PGD-20 last | robust accuracy: 52.80 -> 56.46 | Helped |
| Add consistency regularization from the stable model (ablation) | PGD-AT + Forgetting + Consolidation · CIFAR-10, AutoAttack last | robust accuracy: 49.30 -> 51.28 | Helped |
Surprising findings
- Last-epoch robust accuracy of FOMO consistently exceeds best-epoch robust accuracy of prior early-stopped methods, suggesting early stopping may be unnecessary.
- Forgetting only early layers degrades robust test accuracy and fails to regularize the robust generalization gap, while forgetting later layers mitigates it.
Limitations
- Lacks a theoretical understanding of why forgetting mitigates robust overfitting, as acknowledged by the authors.
- Hyperparameters such as forgetting sparsity and relearning interval require tuning (e.g., 3.5% for PreActResNet-18, 5% for WRN, e_r=5), and the relationship is studied empirically only.
- Evaluation focuses on CIFAR-10/100, SVHN and Tiny-ImageNet with l-inf and some l2 perturbations, with a single architecture for many analyses.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 SVHN Tiny-ImageNet
Benchmarks: CIFAR-10 CIFAR-100 SVHN AutoAttack CIFAR-10-C CW attack Tiny-ImageNet
Metrics: robust test accuracy (PGD-20) natural accuracy best-last robust gap (Delta) trade-off score (harmonic mean of natural and robust accurac mean corruption accuracy
Baselines: PGD-AT TRADES KD+SWA PGD-AT+TE AWP MLCAT IDBH Sub-AT WA
Builds on: PGD adversarial training TRADES AWP KD+SWA temporal ensembling
Closest papers by meaning
- On the Over-Memorization During Natural, Robust and Catastrophic Overfitting (ICLR 2024 · similarity 0.89)
- Conserve-Update-Revise to Cure Generalization and Robustness Trade-off in Adversarial Training (ICLR 2024 · similarity 0.89)
- On the Clean Generalization and Robust Overfitting in Adversarial Training from Two Theoretical Views: Representation Complexity and Training Dynamics (ICML 2025 · similarity 0.88)
- Layer-Aware Analysis of Catastrophic Overfitting: Revealing the Pseudo-Robust Shortcut Dependency (ICML 2024 · similarity 0.87)
- Upweighting Easy Samples in Fine-Tuning Mitigates Forgetting (ICML 2025 · similarity 0.87)
- On Local Overfitting and Forgetting in Deep Neural Networks (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.