AI Paper Atlas

Research map / Label-efficient and robust learning / Recognition and long-tail

ResMem: Learn what you can and memorize the rest

Zitong Yang, Michal Lukasik, Vaishnavh Nagarajan, Zonglin Li, Ankit Singh Rawat, Manzil Zaheer, Aditya Krishna Menon, Sanjiv Kumar

NeurIPS 2023 · Label-efficient and robust learning › Recognition and long-tail · approach: Contrastive representation theory · 3 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Add k-NN residual memorization to CIFAR-ResNet-8CIFAR-ResNet-8 · CIFAR100 testaccuracy: 56.46% -> 59.66%Improved
Replace direct label retrieval with residual retrievalCIFAR-ResNet-8 · CIFAR100 testaccuracy: 54.44% -> 59.66%Improved
Add k-NN residual memorization to T5-smallT5-small · C4 validationprediction accuracy: 38.01% -> 40.87%Improved
Add k-NN residual memorization to T5-largeT5-large · C4 validationprediction accuracy: 44.8% -> 45.6%Improved
Add k-NN residual memorizationMobileNet-V2 models · ImageNet testaccuracy: 60.2% -> 61.2%; 65.3% -> 66.1%; 69.6% -> 70.1%; 71.3% -> 71.8%; 74.7% -> 75.1%Improved
Use direct k-NN label retrieval instead of residual retrievalT5-large · C4 validationprediction accuracy: ResMem accuracy 45.55%; best kNN-LM accuracy 44.88%Hurt
Stop base-network training at 128 epochs rather than training for 256 epochs (ablation)CIFAR-ResNet-8 with ResMem · CIFAR100 testaccuracy: 59.5% -> 49.3%Hurt
Stop base-network training at 160 epochs rather than training for 256 epochs (ablation)CIFAR-ResNet-8 with ResMem · CIFAR100 testaccuracy: 59.5% -> 60.2%Improved
Stop base-network training at 192 epochs rather than training for 256 epochs (ablation)CIFAR-ResNet-8 with ResMem · CIFAR100 testaccuracy: 59.5% -> 58.6%Hurt
Stop base-network training at 224 epochs rather than training for 256 epochs (ablation)CIFAR-ResNet-8 with ResMem · CIFAR100 testaccuracy: 59.5% -> 59.2%Hurt
Vary the number of neighbors across k=27 to k=500 (ablation)ResMem · CIFAR100 testaccuracyNo clear effect
Vary the locality parameter sigma (ablation)ResMem · CIFAR100 testaccuracyMixed

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy