Research map / Continual learning / Continual fine-tuning of LLMs
Shared LoRA Subspaces for almost Strict Continual Learning
Prakhar Kaushik, Ankit Vaidya, Shravan Sunil Chaudhari, Rama Chellappa, Alan Yuille
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| finetune task coefficients after merging (Share-full) | Share (CL) · Continual GLUE average | average score: 78.69 -> 83.44 | Improved |
| continual Share instead of per-task LoRA | LoRA (non-CL) · Continual GLUE average | average score: 83.43 -> 78.69 | Hurt |
| increase temporary factors phi from 1 to 3 (ablation) | Share-full · Continual GLUE RTE | accuracy: 69.675 -> 74.368 | Improved |
| increase temporary factors phi from 3 to 4 (ablation) | Share-full · Continual GLUE RTE | accuracy: 74.368 -> 73.285 | No clear effect |
| increase pseudo-rank p (ablation) | Share-full · Continual GLUE RTE | accuracy | Improved |
| increase number of principal factors k (ablation) | Share-full · MRPC | accuracy | No clear effect |
| share low-rank subspace across tasks, backward transfer | Share · Continual GLUE CoLA | Matthews correlation: 56.00 -> 59.81 | Improved |
| Share instead of TIES merging | TIES (non-CL) · OOD LoRA tasks (Mistral-7B) | Rouge-L: 21.12 -> 55.89 | Improved |
| Share instead of DAP | DAP · CIFAR-100 (10 tasks) | final accuracy: 94.05 -> 94.20 | No clear effect |
| Share (replay-free) instead of iNeMO (replay) | iNeMO · Pascal3D+ | pi/6 accuracy: 79.28 -> 81.80 | Improved |
| Share (replay-free) instead of iNeMO (replay) | iNeMO · Occluded Pascal3D+ L3 | pi/6 accuracy: 34.01 -> 35.50 | Improved |
| Share instead of non-CL LoRA for sequential text-to-image tasks | LoRA (non-CL) · Text-to-image T1 (Flux) | CLIP score: 22.776 -> 23.382 | Improved |
Closest papers by meaning
- C-LoRA: Continual Low-Rank Adaptation for Pre-trained Visual Models (NeurIPS 2026 · similarity 0.94)
- PaLoRA: Paced Low-Rank Adaptation for Continual Learning (NeurIPS 2026 · similarity 0.92)
- CL-LoRA: Continual Low-Rank Adaptation for Rehearsal-Free Class-Incremental Learning (CVPR 2025 · similarity 0.92)
- SD-LoRA: Scalable Decoupled Low-Rank Adaptation for Class Incremental Learning (ICLR 2025 · similarity 0.90)
- LoRA Subtraction for Drift-Resistant Space in Exemplar-Free Continual Learning (CVPR 2025 · similarity 0.89)
- When the Merge Coefficient Stops Mattering: Proximity Regularized Merging for Continual LoRA Adaptation (NeurIPS 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
