Research map / Label-efficient and robust learning / Recognition and long-tail
ResMem: Learn what you can and memorize the rest
Zitong Yang, Michal Lukasik, Vaishnavh Nagarajan, Zonglin Li, Ankit Singh Rawat, Manzil Zaheer, Aditya Krishna Menon, Sanjiv Kumar
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add k-NN residual memorization to CIFAR-ResNet-8 | CIFAR-ResNet-8 · CIFAR100 test | accuracy: 56.46% -> 59.66% | Improved |
| Replace direct label retrieval with residual retrieval | CIFAR-ResNet-8 · CIFAR100 test | accuracy: 54.44% -> 59.66% | Improved |
| Add k-NN residual memorization to T5-small | T5-small · C4 validation | prediction accuracy: 38.01% -> 40.87% | Improved |
| Add k-NN residual memorization to T5-large | T5-large · C4 validation | prediction accuracy: 44.8% -> 45.6% | Improved |
| Add k-NN residual memorization | MobileNet-V2 models · ImageNet test | accuracy: 60.2% -> 61.2%; 65.3% -> 66.1%; 69.6% -> 70.1%; 71.3% -> 71.8%; 74.7% -> 75.1% | Improved |
| Use direct k-NN label retrieval instead of residual retrieval | T5-large · C4 validation | prediction accuracy: ResMem accuracy 45.55%; best kNN-LM accuracy 44.88% | Hurt |
| Stop base-network training at 128 epochs rather than training for 256 epochs (ablation) | CIFAR-ResNet-8 with ResMem · CIFAR100 test | accuracy: 59.5% -> 49.3% | Hurt |
| Stop base-network training at 160 epochs rather than training for 256 epochs (ablation) | CIFAR-ResNet-8 with ResMem · CIFAR100 test | accuracy: 59.5% -> 60.2% | Improved |
| Stop base-network training at 192 epochs rather than training for 256 epochs (ablation) | CIFAR-ResNet-8 with ResMem · CIFAR100 test | accuracy: 59.5% -> 58.6% | Hurt |
| Stop base-network training at 224 epochs rather than training for 256 epochs (ablation) | CIFAR-ResNet-8 with ResMem · CIFAR100 test | accuracy: 59.5% -> 59.2% | Hurt |
| Vary the number of neighbors across k=27 to k=500 (ablation) | ResMem · CIFAR100 test | accuracy | No clear effect |
| Vary the locality parameter sigma (ablation) | ResMem · CIFAR100 test | accuracy | Mixed |
Closest papers by meaning
- The Pitfalls of Memorization: When Memorization Hurts Generalization (ICLR 2025 · similarity 0.87)
- Unveiling Memorization–Generalization Coexistence: A Case Study on Arithmetic Tasks with Label Noise (NeurIPS 2026 · similarity 0.86)
- To Grok or not to Grok: Disentangling Generalization and Memorization on Corrupted Algorithmic Datasets (ICLR 2024 · similarity 0.86)
- Mitigating Memorization in Language Models (ICLR 2025 · similarity 0.86)
- Why Diffusion Models Don’t Memorize: The Role of Implicit Dynamical Regularization in Training (NeurIPS 2025 · similarity 0.85)
- Memorization Sinks: Isolating Memorization during LLM Training (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
