Research map / Continual learning / Online and replay-based
Layerwise Optimization by Gradient Decomposition for Continual Learning
Shixiang Tang, Dapeng Chen, Jinguo Zhu, Shijie Yu, Wanli Ouyang
Check your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the shared-component constraint to the fine-tuned single-predictor baseline (ablation) | Continual learning model · Split CIFAR100 | ACC: 54.7% to 65.3% | Improved |
| Add the shared-component constraint to the fine-tuned single-predictor baseline (ablation) | Continual learning model · Split tinyImageNet | ACC: 21.8% to 33.5% | Improved |
| Add the task-specific component constraint to SCC (ablation) | Continual learning model · Split CIFAR100 | ACC: 65.3% to 67.8% | Improved |
| Add the task-specific component constraint to SCC (ablation) | Continual learning model · Split tinyImageNet | ACC: 33.5% to 36.4% | Improved |
| Add layerwise gradient update to SCC (ablation) | Continual learning model · MNIST Permutation, Split CIFAR10, Split CIFAR100, Split tinyImageNet | ACC: 76.4 -> 82.0; 75.8 -> 76.2; 65.3 -> 67.3; 33.5 -> 36.3 | Improved |
| Add PCA relaxation to SCC+TSCC (ablation) | Continual learning model · Split CIFAR100 | ACC: 67.8 to 68.2 | Improved |
| Add PCA relaxation to SCC+TSCC (ablation) | Continual learning model · Split tinyImageNet | ACC: 36.4 to 38.1 | Improved |
| Increase the rank K of task-specific constraints beyond K=5 (ablation) | PCA-relaxed method · Split CIFAR100 and Split tinyImageNet | ACC | Hurt |
| Increase the rank K of task-specific constraints (ablation) | PCA-relaxed method · Split CIFAR100 and Split tinyImageNet | BWT | Improved |
| Add layerwise gradient update (ablation) | GEM · MNIST | ACC: 81.9 -> 82.7 | Improved |
| Add layerwise gradient update (ablation) | GEM · Split CIFAR100 | BWT: -0.005 -> 0.003 | Improved |
| Add layerwise gradient update (ablation) | A-GEM · MNIST | BWT: 0.005 -> -0.008 | Hurt |
Closest papers by meaning
- On the Theory of Continual Learning with Gradient Descent for Neural Networks (ICML 2026 · similarity 0.91)
- Continual Learning with Scaled Gradient Projection (AAAI 2023 · similarity 0.90)
- Growing a Brain with Sparsity-Inducing Generation for Continual Learning (ICCV 2023 · similarity 0.90)
- Train with Perturbation, Infer after Merging: A Two-Stage Framework for Continual Learning (NeurIPS 2025 · similarity 0.89)
- Convergence and Implicit Bias of Gradient Descent on Continual Linear Classification (ICLR 2025 · similarity 0.89)
- An Effective Dynamic Gradient Calibration Method for Continual Learning (ICML 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
