Research map / Architectures and efficiency / CNN design
Identity Crisis: Memorization and Generalization Under Extreme Overparameterization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a shallow CNN rather than an FCN | Networks trained on one MNIST example · MNIST identity mapping | generalization to the identity function | Helped |
| Increase CNN depth (ablation) | CNN trained on one MNIST example · MNIST identity mapping | similarity to the identity function | Mixed |
| Increase the training-image size (ablation) | CNN trained on one example · MNIST identity mapping | bias toward the constant function | Helped |
| Increase convolution filter size (ablation) | CNN trained on one MNIST example · MNIST identity mapping | similarity to the identity function | Hurt |
| Use filters covering the entire input (ablation) | CNN trained on one MNIST example · MNIST identity mapping | bias toward the constant function | Helped |
| Use 3 channels instead of a larger channel count (ablation) | CNN trained on one MNIST example · MNIST identity mapping | preservation of image content | Hurt |
| Increase channels to 1024 (ablation) | 5-layer CNN trained on one MNIST example · MNIST identity mapping | overfitting | No clear effect |
| Increase feature-channel count by an order of magnitude (ablation) | 20-layer CNN trained on one MNIST example · MNIST identity mapping | information flow through layers | No clear effect |
| Train both layers instead of only the bottom layer (ablation) | Two-layer ReLU FCN · MNIST identity mapping | prediction behavior on unseen examples | Mixed |
| Increase depth in a fully connected network (ablation) | FCN trained on one example · MNIST identity mapping | prediction behavior on unseen examples | Mixed |
| Train a multilayer linear FCN instead of a one-layer linear FCN (ablation) | Linear FCN trained on one example · MNIST identity mapping | bias toward a constant function | Helped |
| Test a 5-layer CNN on inputs smaller than its training image (ablation) | 5-layer CNN trained on a 28×28 image · Rescaled input images | preservation of the identity mapping | Hurt |
Closest papers by meaning
- Unveiling Memorization–Generalization Coexistence: A Case Study on Arithmetic Tasks with Label Noise (NeurIPS 2026 · similarity 0.90)
- To Grok or not to Grok: Disentangling Generalization and Memorization on Corrupted Algorithmic Datasets (ICLR 2024 · similarity 0.90)
- The Geometric Inductive Bias of Grokking: Bypassing Phase Transitions via Architectural Topology (NeurIPS 2026 · similarity 0.88)
- From Linear to Nonlinear: Provable Weak-to-Strong Generalization through Feature Learning (NeurIPS 2025 · similarity 0.87)
- On the Over-Memorization During Natural, Robust and Catastrophic Overfitting (ICLR 2024 · similarity 0.87)
- Differential learning kinetics govern the transition from memorization to generalization during in-context learning (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.