Research map / Continual learning / Class-incremental
Remember the Past: Distilling Datasets into Addressable Memories for Neural Networks
Zhiwei Deng, Olga Russakovsky
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add shared bases with learned addressing matrices (ablation) | Ours (memory addressing + BPTT) · CIFAR10 1 img/class | accuracy (%): 55.2 -> 64.2 | Improved |
| add downsampled bases (ablation) | Ours (memory addressing + BPTT) · CIFAR10 1 img/class | accuracy (%): 49.1 -> 55.2 | Improved |
| add data augmentation (ablation) | Ours (memory addressing + BPTT) · CIFAR10 1 img/class | accuracy (%): 64.2 -> 66.4 | Improved |
| add momentum in inner loop (ablation) | Ours (memory addressing + BPTT) · CIFAR10 1 img/class BPTT | accuracy (%): 40.5 -> 49.1 | Improved |
| pass momentum gradients through backward pass (full vs forward-only) (ablation) | Ours (memory addressing + BPTT) · CIFAR10 1 img/class BPTT | accuracy (%): 45.6 -> 49.1 | Improved |
| use longer unrolled inner loop (vs 1 step) (ablation) | Ours (memory addressing + BPTT) · CIFAR10 BPTT | accuracy (%): +18.2 | Improved |
| use BPTT instead of single-step gradient matching (ablation) | Dataset distillation · CIFAR10 1 img/class | accuracy (%): 28.8 -> 49.1 | Improved |
| use Adam instead of SGD with momentum for inner loop (ablation) | Ours (memory addressing + BPTT) · distillation | accuracy (%) | No clear effect |
| distill with memory addressing instead of per-task buffer for continual learning | compress-then-recall · MANY Permutations | retained accuracy: 50.81 -> 74.07 | Improved |
| address memories with image features instead of labels (vs nearest neighbor) | Ours (memory addressing + BPTT) · CIFAR100 5-shot | accuracy (%): 61.72 -> 71.20 | Improved |
| image addressing vs classify-then-recall | Ours (memory addressing + BPTT) · CIFAR100 5-shot | accuracy (%): 58.46 -> 71.20 | Improved |
Closest papers by meaning
- Dataset Distillation as Data Compression: A Rate-Utility Perspective (ICCV 2025 · similarity 0.89)
- Large Scale Dataset Distillation with Domain Shift (ICML 2024 · similarity 0.88)
- Boost Self-Supervised Dataset Distillation via Parameterization, Predefined Augmentation, and Approximation (ICLR 2025 · similarity 0.86)
- Data Distillation Can Be Like Vodka: Distilling More Times For Better Quality (ICLR 2024 · similarity 0.86)
- Going Beyond Feature Similarity: Effective Dataset distillation based on Class-aware Conditional Mutual Information (ICLR 2025 · similarity 0.86)
- Dataset Distillation via Knowledge Distillation: Towards Efficient Self-Supervised Pre-training of Deep Networks (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
