Research map / Adversarial robustness and security / Certified and empirical robustness
On the Over-Memorization During Natural, Robust and Catastrophic Overfitting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Distraction Over-Memorization (DOM), which detects over-memorized training patterns via natural-pattern loss and removes or augments them, mitigating natural, robust, and catastrophic overfitting.
Problem
Overfitting in natural training (natural overfitting), multi-step adversarial training (robust overfitting), and single-step adversarial training (catastrophic overfitting) is typically handled separately, without a shared understanding or unified remedy.
Approach
Identifies a shared "over-memorization" behaviour: the model suddenly becomes high-confidence on certain training patterns and retains persistent memory of them; over-memorized adversarial patterns coincide with high-confidence natural patterns, so detection needs only the natural training loss. DOM removes (DOM_RE) or iteratively augments (DOM_DA) high-confidence natural-loss samples.
Key results
- On CIFAR-10 with PreactResNet-18 natural training, DOM_DA with RandAugment reduces last-checkpoint test error from 4.31% to 4.07% and shrinks the generalization gap.
- In single-step AT on CIFAR-10 (RS-FGSM), DOM_RE raises final-checkpoint PGD-20 accuracy from 0.00% to 47.09% and AutoAttack accuracy from 0.00% to 17.10%, mitigating catastrophic overfitting.
- DOM_DA (AugMix) under multi-step AT improves CIFAR-10 last-checkpoint PGD-20 accuracy from 45.16% to 49.31% and AutoAttack from 42.70% to 45.51%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Remove high-confidence natural patterns after warm-up | Natural training with PreactResNet-18 · CIFAR-10 | last test error: 4.84 ± 0.04 -> 4.63 ± 0.19 | Helped |
| Iteratively augment high-confidence patterns with AUGMIX | Natural training with PreactResNet-18 · CIFAR-10 | last test error: 4.52 ± 0.01 -> 4.24 ± 0.02 | Helped |
| Iteratively augment high-confidence patterns with RandAugment | Natural training with PreactResNet-18 · CIFAR-100 | last test error: 21.61 ± 0.11 -> 21.49 ± 0.06 | Helped |
| Remove high-confidence natural patterns after warm-up | Multi-step adversarial training · CIFAR-10 | PGD-20 accuracy, last checkpoint: 45.16 ± 1.20 -> 52.52 ± 1.29 | Helped |
| Remove high-confidence natural patterns after warm-up | Multi-step adversarial training · CIFAR-10 | Auto Attack accuracy, last checkpoint: 42.70 ± 1.16 -> 32.90 ± 1.02 | Hurt |
| Iteratively augment high-confidence patterns with AUGMIX | Multi-step adversarial training · CIFAR-10 | Auto Attack accuracy, last checkpoint: 44.71 ± 1.06 -> 45.51 ± 0.85 | Helped |
| Iteratively augment high-confidence patterns with RandAugment | Multi-step adversarial training · CIFAR-10 | Auto Attack accuracy, last checkpoint: 44.27 ± 1.63 -> 46.62 ± 0.69 | Helped |
| Remove high-confidence natural patterns after warm-up | Single-step adversarial training · CIFAR-100 | PGD-20 accuracy: 0.00 ± 0.00 -> 12.68 ± 0.62 | Helped |
| Use AUGMIX iterative augmentation rather than baseline AUGMIX alone | Single-step adversarial training · CIFAR-100 | PGD-20 accuracy: 0.00 ± 0.00 -> 0.00 ± 0.00 | No clear effect |
| Use RandAugment iterative augmentation rather than baseline RandAugment alone | Single-step adversarial training · CIFAR-100 | PGD-20 accuracy: 0.00 ± 0.00 -> 27.44 ± 0.12 | Helped |
| Set loss threshold too small or too large (ablation) | DOM_RE · Natural, multi-step AT, and single-step AT | test error or robustness | Mixed |
| Shorten or lengthen the warm-up period from the selected timing (ablation) | DOM_RE · Natural training | generalization performance | Hurt |
Surprising findings
- Over-memorized adversarial patterns show about 90% overlap with high-confidence natural patterns, so natural training loss alone can identify them.
- DOM_RE (removing high-confidence samples) reduces Auto Attack robustness under multi-step AT on CIFAR-10 relative to the baseline (32.90% vs 42.70% at last checkpoint).
Limitations
- No detailed theoretical analysis of the mechanisms linking natural, robust, and catastrophic overfitting.
- DOM_DA effectiveness depends on the quality of the underlying data augmentation; it fails to prevent catastrophic overfitting with AugMix on CIFAR-100.
- Loss threshold must be tuned per paradigm and dataset; higher thresholds can increase vulnerability to AutoAttack.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 SVHN Tiny-ImageNet
Benchmarks: CIFAR-10 CIFAR-100 SVHN Tiny-ImageNet
Metrics: test error (natural) generalization gap (Diff) natural accuracy PGD-20 robust accuracy AutoAttack robust accuracy
Baselines: Baseline (standard training without DOM) AugMix RandAugment adaptive loss threshold
Builds on: PGD adversarial training RS-FGSM TRADES AugMix RandAugment AutoAttack
Closest papers by meaning
- The Effectiveness of Random Forgetting for Robust Generalization (ICLR 2024 · similarity 0.89)
- On Local Overfitting and Forgetting in Deep Neural Networks (AAAI 2025 · similarity 0.88)
- Rethinking Benign Overfitting in Two-Layer Neural Networks (ICML 2025 · similarity 0.87)
- The Pitfalls of Memorization: When Memorization Hurts Generalization (ICLR 2025 · similarity 0.87)
- Unveiling Memorization–Generalization Coexistence: A Case Study on Arithmetic Tasks with Label Noise (NeurIPS 2026 · similarity 0.87)
- How Spurious Features are Memorized: Precise Analysis for Random and NTK Features (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.