Research map / Adversarial robustness and security / Certified and empirical robustness
A Closer Look at Memorization in Deep Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train on real data instead of random inputs or labels | DNNs trained with SGD · MNIST and CIFAR-10 | critical sample ratio | Helped |
| Include shared patterns in the training data rather than using fully random data | DNNs trained with SGD · Real data versus randX and randY | example difficulty variation after one epoch | Mixed |
| Increase model capacity when training with noisy inputs | 2-layer MLPs · MNIST with noisy inputs | validation performance | Helped |
| Increase model capacity | 2-layer MLPs · CIFAR-10 with random labels | validation performance | No clear effect |
| Replace more training inputs or labels with noise | DNNs trained with SGD · MNIST and CIFAR-10 | critical sample ratio | Hurt |
| Replace more training inputs or labels with noise | DNNs trained with SGD · MNIST and CIFAR-10 | validation accuracy | Hurt |
| Train with random labels alongside real data | DNNs trained with SGD · MNIST and CIFAR-10 | validation accuracy during training | Mixed |
| Add dropout regularization | CNN · CIFAR-10 and randY | randY training accuracy at preserved CIFAR-10 validation accuracy | Helped |
| Add dropout with adversarial training | CNN · CIFAR-10 and randY | randY training accuracy at preserved CIFAR-10 validation accuracy | Helped |
| Use regularization to slow fitting random labels | CNN · CIFAR-10 and randY | memorization speed | Helped |
Closest papers by meaning
- To Grok or not to Grok: Disentangling Generalization and Memorization on Corrupted Algorithmic Datasets (ICLR 2024 · similarity 0.90)
- The Pitfalls of Memorization: When Memorization Hurts Generalization (ICLR 2025 · similarity 0.89)
- Unveiling Memorization–Generalization Coexistence: A Case Study on Arithmetic Tasks with Label Noise (NeurIPS 2026 · similarity 0.89)
- On the Over-Memorization During Natural, Robust and Catastrophic Overfitting (ICLR 2024 · similarity 0.89)
- Generalizablity of Memorization Neural Network (NeurIPS 2024 · similarity 0.88)
- Towards Memorization Estimation: Fast, Formal and Free (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.