Research map / Continual learning / Continual fine-tuning of LLMs
Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgetting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use explicit task-specific structure search with reuse, adaptation, and new-layer choices | Learn-to-Grow · Permuted MNIST (10 tasks) | average accuracy (%): 98.48 vs 97.98 | Helped |
| Use explicit task-specific structure search with reuse, adaptation, and new-layer choices | Learn-to-Grow · Split CIFAR-100 (10 tasks) | average accuracy (%): 76.21 vs 74.52 | Helped |
| Use learned structure growth rather than the Adapter baseline | Learn-to-Grow · Visual Domain Decathlon | average top-1 validation accuracy (%): 77.68 vs 76.02 | Helped |
| Keep reused layers fixed during retraining (ablation) | Learn-to-Grow · Permuted MNIST (10 tasks) | average accuracy (%): 98.29 -> 98.46 | Helped |
| Add L2 regularization when fine-tuning reused layers (ablation) | Learn-to-Grow · Permuted MNIST (10 tasks) | average accuracy (%): 98.29 -> 98.48 | Helped |
| Keep reused layers fixed during retraining (ablation) | Learn-to-Grow · Split CIFAR-100 (10 tasks) | average accuracy (%): 75.05 -> 75.31 | Helped |
| Add L2 regularization when fine-tuning reused layers (ablation) | Learn-to-Grow · Split CIFAR-100 (10 tasks) | average accuracy (%): 75.05 -> 76.21 | Helped |
| Force new parameters in the first layer and reuse the remaining layers (ablation) | Learn-to-Grow · Permuted MNIST | average accuracy over seen tasks | Helped |
| Use new parameters in a different layer instead of the learned first-layer choice (ablation) | Learn-to-Grow · Permuted MNIST | average accuracy over seen tasks | Hurt |
| Fine-tune reused layers with a smaller learning rate and no regularization (ablation) | Learn-to-Grow · Permuted MNIST | overall accuracy | No clear effect |
| Fine-tune all shared backbone layers sequentially instead of learning task-specific structures (ablation) | Visual Domain Decathlon · Visual Domain Decathlon | classification accuracy | Hurt |
Closest papers by meaning
- Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks (ICML 2025 · similarity 0.90)
- Do Your Best and Get Enough Rest for Continual Learning (CVPR 2025 · similarity 0.90)
- Predicting the Susceptibility of Examples to Catastrophic Forgetting (ICML 2025 · similarity 0.90)
- On the Diminishing Returns of Width for Continual Learning (ICML 2024 · similarity 0.89)
- STAR: Stability-Inducing Weight Perturbation for Continual Learning (ICLR 2025 · similarity 0.89)
- Kolmogorov-Arnold Networks Still Catastrophically Forget but Differently from MLP (AAAI 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.