Research map / Architectures and efficiency / RNNs and normalization
Can Neural Network Memorization Be Localized?
Pratyush Maini, Michael Curtis Mozer, Hanie Sedghi, Zachary Chase Lipton, J Zico Kolter, Chiyuan Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add example-tied dropout (memorization neurons dropped at test time) | ResNet-9 · CIFAR-10 10% label noise | noisy train accuracy after dropout: 99.3% -> 3.1% | Improved |
| add example-tied dropout (memorization neurons dropped at test time) | ResNet-9 · MNIST 10% label noise | noisy train accuracy after dropout: 100% -> 0.1% | Improved |
| add example-tied dropout (memorization neurons dropped at test time) | ResNet-9 · SVHN 10% label noise | noisy train accuracy after dropout: 99.6% -> 1.4% | Improved |
| add example-tied dropout: clean-example accuracy cost after memorization neuron drop | ResNet-9 · CIFAR-10 10% label noise | clean train accuracy: 99.9% -> 90.8% | Hurt |
| add example-tied dropout: test accuracy after memorization neuron drop | ResNet-9 · CIFAR-10 10% label noise | test accuracy: 79.3% -> 82.7% | Mixed |
| grid over p_mem and p_gen for example-tied dropout (ablation) | ResNet-9 · CIFAR-10 10% label noise | noisy train accuracy after dropout: 2.8% to 6.9% across grid | Mixed |
| use standard dropout (p=0.4) instead of example-tied dropout (ablation) | ResNet-9 · CIFAR-10 10% label noise | noisy train accuracy: 99.5% vs 3.10% for example-tied | Hurt |
| use neuron-removal count as mislabeled-example detector instead of learning time or forgetting time | ResNet-9 · CIFAR-10 10% label noise detection | AUC: 0.89 vs 0.87 (learning time) and 0.93 (forgetting time) | Mixed |
| retrain rewound layers on clean examples only (layer retraining) vs. memorization in last layers | ResNet-9 · CIFAR-10 and SVHN 10% label noise | noisy example accuracy after clean-only retraining | No clear effect |
| rewind final layers to earlier checkpoints (ablation) | ResNet-9 / ResNet-50 / ViT · CIFAR-10 noisy training examples | accuracy on memorized examples | No clear effect |
Closest papers by meaning
- The Pitfalls of Memorization: When Memorization Hurts Generalization (ICLR 2025 · similarity 0.89)
- Unveiling Memorization–Generalization Coexistence: A Case Study on Arithmetic Tasks with Label Noise (NeurIPS 2026 · similarity 0.89)
- Why Diffusion Models Don’t Memorize: The Role of Implicit Dynamical Regularization in Training (NeurIPS 2025 · similarity 0.88)
- Towards Memorization Estimation: Fast, Formal and Free (ICML 2025 · similarity 0.88)
- On the Over-Memorization During Natural, Robust and Catastrophic Overfitting (ICLR 2024 · similarity 0.88)
- A mathematical theory of balancing relational generalization and memorization (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
