AI Paper Atlas

Research map / Continual learning / Online and replay-based

Is Forgetting Less a Good Inductive Bias for Forward Transfer?

Jiefeng Chen, Timothy Nguyen, Dilan Gorur, Arslan Chaudhry

ICLR 2023 · Continual learning › Online and replay-based · approach: Theoretical analysis of forgetting · 1 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Retain more knowledge from previous tasks during continual trainingContinual learning representations · Split CIFAR-100, random initK-shot forward transfer: 0.92 (Spearman correlation, k=5)Improved
Retain more knowledge from previous tasks during continual trainingContinual learning representations · Split CIFAR-10, random initK-shot forward transfer: 0.64 (k=5); 0.56 (k=10); 0.65 (k=20)Improved
Retain more knowledge from previous tasks during continual trainingContinual learning representations · CLEAR100, pretrainedK-shot forward transfer: 0.83 (k=5); 0.8 (k=10); 0.81 (k=20)Improved
Retain more knowledge from previous tasks during continual trainingContinual learning representations · CIFAR-100 Superclasses, pretrained, k=5K-shot forward transfer: 0.16 (0.40)No clear effect
Use LP-FT, which adds linear probing before finetuning each taskImageNet-pretrained representations · Split CIFAR-10 and Split CIFAR-100K-shot forward transferMixed
Use ResNet18 instead of ResNet50 (ablation)Continual learning representations · Split CIFAR-10, Split CIFAR-100, and CIFAR-100 SuperclassesRelationship between forgetting and forward transferNo clear effect
Measure transfer with auxiliary k-shot evaluation on fixed representations rather than accuracy after constrained continual updatesForward-transfer evaluation · Image classification benchmarksAssociation between knowledge retention and forward transfer: Most entries are above 0.5Improved
Retain less information from previous tasksContinual learning representations · CIFAR-100 Superclasses, pretrained, k=5K-shot forward transfer: 0.16 (0.40)No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy