AI Paper Atlas

Research map / Continual learning / Online and replay-based

Gradient based sample selection for online continual learning

Rahaf Aljundi, Min Lin, Baptiste Goujaud, Yoshua Bengio

NeurIPS 2019 · Continual learning › Online and replay-based · approach: Replay buffer construction and update · 76 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use gradient-diversity greedy selection instead of random buffer selectiononline continual learning · Disjoint MNIST, buffer 300average test accuracy: 37.5±1.3 -> 82.6±2.9Improved
Use gradient-diversity greedy selection instead of random buffer selectiononline continual learning · Disjoint CIFAR-10average test accuracy: 28.6±1.2 -> 33.56± 1.7Improved
Select buffer samples with gradient-based methods rather than reservoir samplingonline continual learning · Imbalanced disjoint MNIST sequencesaverage accuracy: 69.12±4.3 -> 77.96± 3.5Improved
Use gradient-space clustering rather than feature-space clusteringonline continual learning · Permuted MNISTaverage test accuracy: 77.8 ± 0.3 -> 79.74±0.2Improved
Use greedy gradient selection rather than task-aware GEMonline continual learning · Disjoint CIFAR-10average test performanceImproved
Use constrained optimization updates instead of rehearsal updates (ablation)GSS-IQP · Disjoint MNIST, buffer 100average test accuracy: 71.3 -> 76.26Improved
Use constrained optimization updates instead of rehearsal updates (ablation)GSS-IQP · Disjoint MNIST, buffer 200average test accuracy: 78.9 -> 83.4Improved
Increase the number of sampled buffer gradients used to estimate similarity from n=1 to n=10 (ablation)GSS-Greedy · Disjoint MNISTaverage test accuracy: Avg. Acc.=67.3 for n=1Improved
Increase n to large values in greedy gradient similarity scoring (ablation)GSS-Greedy · Disjoint MNISTaverage test accuracyHurt
Increase batch size (ablation)GSS-Greedy · Disjoint MNISTaverage test accuracyHurt
Use a very small batch size (ablation)GSS-Greedy · Disjoint MNISTaverage test accuracyMixed
Increase rehearsal buffer size from 100 to 200 (ablation)GSS-IQP (Rehearsal) · Disjoint MNISTaverage test accuracy: 71.3 -> 78.9Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy