AI Paper Atlas

Research map / Continual learning / Online and replay-based

Layerwise Optimization by Gradient Decomposition for Continual Learning

Shixiang Tang, Dapeng Chen, Jinguo Zhu, Shijie Yu, Wanli Ouyang

CVPR 2021 · Continual learning › Online and replay-based · approach: Gradient projection and plasticity preservation · 62 citations (OpenAlex)

Check your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Add the shared-component constraint to the fine-tuned single-predictor baseline (ablation)Continual learning model · Split CIFAR100ACC: 54.7% to 65.3%Improved
Add the shared-component constraint to the fine-tuned single-predictor baseline (ablation)Continual learning model · Split tinyImageNetACC: 21.8% to 33.5%Improved
Add the task-specific component constraint to SCC (ablation)Continual learning model · Split CIFAR100ACC: 65.3% to 67.8%Improved
Add the task-specific component constraint to SCC (ablation)Continual learning model · Split tinyImageNetACC: 33.5% to 36.4%Improved
Add layerwise gradient update to SCC (ablation)Continual learning model · MNIST Permutation, Split CIFAR10, Split CIFAR100, Split tinyImageNetACC: 76.4 -> 82.0; 75.8 -> 76.2; 65.3 -> 67.3; 33.5 -> 36.3Improved
Add PCA relaxation to SCC+TSCC (ablation)Continual learning model · Split CIFAR100ACC: 67.8 to 68.2Improved
Add PCA relaxation to SCC+TSCC (ablation)Continual learning model · Split tinyImageNetACC: 36.4 to 38.1Improved
Increase the rank K of task-specific constraints beyond K=5 (ablation)PCA-relaxed method · Split CIFAR100 and Split tinyImageNetACCHurt
Increase the rank K of task-specific constraints (ablation)PCA-relaxed method · Split CIFAR100 and Split tinyImageNetBWTImproved
Add layerwise gradient update (ablation)GEM · MNISTACC: 81.9 -> 82.7Improved
Add layerwise gradient update (ablation)GEM · Split CIFAR100BWT: -0.005 -> 0.003Improved
Add layerwise gradient update (ablation)A-GEM · MNISTBWT: 0.005 -> -0.008Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy