Research map / Continual learning / Continual fine-tuning of LLMs
Scalable Language Model with Generalized Continual Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Scalable Language Model (SLM), combining Joint Adaptive Re-Parameterization (JARe) with Dynamic Task-related Knowledge Retrieval (DTKR) so that LoRA-style weight increments are stored as key-value pairs and retrieved by query-key similarity for continual learning without replay, optimization constraints, or task-ID.
Problem
Continual learning for language models typically relies on experience replay, optimization constraints, or task-IDs at inference, which are impractical in real-world settings; most prior work also covers only single task types such as classification.
Approach
Store each task's low-rank weight increment as a value paired with a learned key that estimates the task distribution centroid; at inference, retrieve the K nearest keys to the input's query (Sentence-BERT features) and re-parameterize the frozen pretrained model with a similarity-weighted average of the retrieved increments. Group-based retrieval and random key masking address keys getting stuck
Key results
- On the BERT benchmark (order 4-7 average), SLM reaches 79.1% average accuracy without task-ID or replay, versus 76.3% for IDBR (with replay) and 77.9% for ProgPrompt (task-ID); SLM-TI reaches 80.0%.
- On the T5 few-shot benchmark, SLM averages 73.1% versus 52.7% for LFPT5 (which uses slight replay).
- On the LLaMA-2-7B benchmark, SLM averages 82.3% across two task orders versus 38.5% for fine-tuning and 62.3% for replay.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use SLM instead of IDBR | BERT-base continual learning · BERT benchmark (Orders 4–7, Avg) | accuracy: 76.3 -> 79.1 | Helped |
| use SLM instead of LFPT5 | T5-large few-shot continual learning · T5 benchmark (Orders 1–3, Avg) | accuracy: 52.7 -> 73.1 | Helped |
| use SLM instead of Finetune | LLaMA2-7B continual learning · LLaMA benchmark (Finance → MMLU → Medical), Finance | accuracy: 18.0 -> 89.0 | Helped |
| replace JARe weight increments with prompts (ablation) | DTKR on BERT-base · BERT benchmark (Orders 4–6, Avg) | accuracy: 79.0 -> 53.3 | Hurt |
| replace JARe weight increments with adapters (ablation) | DTKR on BERT-base · BERT benchmark (Orders 4–6, Avg) | accuracy: 79.0 -> 70.9 | Hurt |
| fine-tune tasks individually instead of continual learning (ablation) | BERT-base · BERT benchmark (Orders 4–6, Avg) | accuracy: 79.0 -> 79.8 | Helped |
| use out-layer attention weights rather than query, key, or value weights (ablation) | BERT-base re-parameterization · BERT tasks (amazon, ag, yahoo, db; Average) | accuracy: 82.3 -> 83.1 | Helped |
| remove random keys mask (ablation) | group-based key retrieval on BERT-base · BERT benchmark (key retrieval accuracy, Avg) | accuracy: 96.1 -> 94.6 | Hurt |
| remove group retrieval and random keys mask (ablation) | key retrieval on BERT-base · BERT benchmark (key retrieval accuracy, Avg) | accuracy: 96.1 -> 89.5 | Hurt |
| increase Rank from 2 to 4 (ablation) | SLM on BERT-base · BERT benchmark (parameter-size ablation, Avg) | accuracy: 77.8 -> 78.7 | Helped |
| increase TopK from 2 to 4 (ablation) | SLM on BERT-base · BERT benchmark (parameter-size ablation, Avg) | accuracy: 78.7 -> 79.1 | Helped |
| increase Rank from 8 to 12 (ablation) | SLM on BERT-base · BERT benchmark (parameter-size ablation, Avg) | accuracy: 79.7 -> 80.0 | Helped |
Surprising findings
- Separate per-task fine-tuning is only 0.8 points above the continual SLM on the BERT benchmark, indicating little gap to the non-continual upper bound.
- Replacing JARe with prompts or adapters in the same DTKR framework drops BERT average accuracy to 53.3% (prompt) and 70.9% (adapter).
Limitations
- Retrieval framework adds computational and memory storage cost that grows with the number of tasks.
- Zero-shot results show mixed effects, e.g. medical zero-shot ARC-e drops well below fine-tuning baseline (44.3 vs 73.2) in the appendix table.
- LLaMA experiments use only two task orders and three tasks due to resource constraints.
Experimental setup
Datasets: AG News Yelp DBPedia Amazon Yahoo Answers MMLU Medical QA (ChatDoctor) Finance sentiment (tweets)
Benchmarks: AG News Yelp DBPedia Amazon Yahoo Answers MMLU Medical QA Finance sentiment ARC-c ARC-e PIQA WinoGrande
Metrics: accuracy BERTScore average accuracy forgetting
Baselines: Finetune Replay MBPA++ IDBR LFPT5 ProgPrompt EWC Prompt
Builds on: LoRA IDBR LFPT5 Sentence-BERT ProgPrompt
Closest papers by meaning
- Learn more, but bother less: parameter efficient continual learning (NeurIPS 2024 · similarity 0.92)
- Train-Attention: Meta-Learning Where to Focus in Continual Knowledge Learning (NeurIPS 2024 · similarity 0.90)
- Spurious Forgetting in Continual Learning of Language Models (ICLR 2025 · similarity 0.90)
- CMT: A Memory Compression Method for Continual Knowledge Learning of Large Language Models (AAAI 2025 · similarity 0.89)
- Interactive Continual Learning: Fast and Slow Thinking (CVPR 2024 · similarity 0.88)
- Data Efficient Adaptation in Large Language Models via Continuous Low-Rank Fine-Tuning (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.