Research map / Continual learning / Continual fine-tuning of LLMs
DLCFT: Deep Linear Continual Fine-Tuning for General Incremental Learning
Hyounguk Shon, Janghyeon Lee, Seung Hwan Kim, Junmo Kim
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add linearization and MSE loss (ablation) | K-FAC regularization · Seq-CIFAR-100 data-IL (10 tasks) | final accuracy: 79.23 -> 81.95 | Improved |
| add linearization and MSE loss (ablation) | K-FAC regularization · Seq-CIFAR-100 data-IL (100 tasks) | final accuracy: 73.51 -> 75.92 | Improved |
| add linearization and MSE loss (ablation) | K-FAC regularization · Seq-CIFAR-100 class-IL (10 tasks) | accuracy: 44.93 -> 59.86 | Improved |
| add linearization and MSE loss (ablation) | TK-FAC regularization · Seq-CIFAR-100 data-IL (10 tasks) | final accuracy: 79.58 -> 82.70 | Improved |
| add linearization and MSE loss (ablation) | TK-FAC regularization · Seq-CIFAR-100 data-IL (100 tasks) | final accuracy: 73.18 -> 80.07 | Improved |
| add linearization and MSE loss (ablation) | TK-FAC regularization · Seq-CIFAR-100 class-IL (10 tasks) | accuracy: 44.66 -> 59.98 | Improved |
| use TK-FAC instead of K-FAC curvature on nonlinear network with SCE (ablation) | nonlinear + SCE · Seq-CIFAR-100 data-IL (10 tasks) | final accuracy: 79.23 -> 79.58 | No clear effect |
| use TK-FAC instead of K-FAC curvature on linearized MSE network (ablation) | DLCFT · Seq-CIFAR-100 data-IL (100 tasks) | final accuracy: 75.92 -> 80.07 | Improved |
| use linearized MSE model with diagonal EWC curvature instead of nonlinear SCE (ablation) | EWC · Seq-CIFAR-100 data-IL (10 tasks) | final accuracy: 78.88 -> 82.17 | Improved |
| add DLCFT regularization compared with no regularization | fine-tuned ResNet-18 · Seq-CIFAR-100 data-IL (100 tasks) | final accuracy: 71.83 -> 80.07 | Improved |
| add LwF regularization compared with no regularization | fine-tuned ResNet-18 · Seq-CIFAR-100 data-IL (100 tasks) | final accuracy: 71.83 -> 70.95 | Hurt |
| add MAS regularization compared with no regularization | fine-tuned ResNet-18 · Seq-CIFAR-100 data-IL (100 tasks) | final accuracy: 71.83 -> 52.50 | Hurt |
Closest papers by meaning
- Compacting, Picking and Growing for Unforgetting Continual Learning (NeurIPS 2019 · similarity 0.88)
- SAFE: Slow and Fast Parameter-Efficient Tuning for Continual Learning with Pre-Trained Models (NeurIPS 2024 · similarity 0.87)
- BiLoRA: Almost-Orthogonal Parameter Spaces for Continual Learning (CVPR 2025 · similarity 0.87)
- Learning Continually by Spectral Regularization (ICLR 2025 · similarity 0.86)
- A Statistical Theory of Regularization-Based Continual Learning (ICML 2024 · similarity 0.86)
- Continual Learning Using a Kernel-Based Method Over Foundation Models (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
