Research map / Continual learning / Continual fine-tuning of LLMs
Spurious Forgetting in Continual Learning of Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Identifies "spurious forgetting" in continual learning of LLMs, attributing performance drops to lost task alignment via orthogonal early-step weight updates, and proposes Freeze (freezing bottom layers) which mitigates it without stored old data.
Problem
LLMs in continual learning show sharp performance drops on previous tasks, which the authors argue reflect loss of task alignment rather than loss of underlying knowledge.
Approach
Performance drops after learning new tasks are largely spurious: early optimization steps undo prior task alignment via near-orthogonal weight updates concentrated in bottom layers. Freezing the bottom layers during fine-tuning lets the model reuse existing alignment without storing old data.
Key results
- On the Biography dataset, Freeze (n_layer=7, early stop) raises Task 0 accuracy from 11.18% (SEQ) to 44.22% while keeping Task 1 at 99.93%, versus 76.93% for REPLAY with 20% old data.
- In safety alignment, Freeze (6 layers) reduces jailbreak rate from 99.80% (SEQ) to 1.15%.
- In continual knowledge editing (ZSRE), Freeze (2 layers, 1 task) improves Efficacy from 62.47 to 70.65 and Paraphrase from 58.24 to 68.60 over SEQ.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| freeze bottom 7 layers and use early stopping instead of sequential fine-tuning | Biography continual learning · Biography Task 0 | ACC: 11.18 -> 44.22 | Helped |
| replay 50% of old data instead of sequential fine-tuning | Biography continual learning · Biography Task 0 | ACC: 11.18 -> 80.62 | Helped |
| freeze bottom 6 layers instead of sequential fine-tuning | Safety alignment · Safety Alignment (SA) | Jailbreak Rate: 99.80 -> 1.15 | Helped |
| freeze bottom 3 layers instead of sequential fine-tuning | Continual instruction tuning · Continual Instruction Tuning (CIT, TRACE) | Test Score: 47.38 -> 53.20 | Helped |
| freeze bottom 1 layer after the first task instead of sequential fine-tuning | Continual knowledge editing · Continual Knowledge Editing (CKE, ZSRE) | Efficacy: 62.47 -> 70.88 | Helped |
| freeze bottom 2 layers after the first task instead of sequential fine-tuning | Continual knowledge editing · Continual Knowledge Editing (CKE, ZSRE) | Paraphrase: 58.24 -> 68.60 | Helped |
| freeze bottom 2 layers after the first task instead of sequential fine-tuning | Instance incremental learning · Instance Incremental Learning (IIL, Concept-1K) | Mem. Acc.: 35.98 -> 42.18 | Helped |
| freeze bottom 2 layers after the first task instead of sequential fine-tuning | Instance incremental learning · Instance Incremental Learning (IIL, Concept-1K) | Gen. Acc.: 12.61 -> 14.19 | Helped |
| add early stopping to Freeze with 8 frozen layers (ablation) | Biography continual learning · Biography Task 0 | ACC: 39.68 -> 42.46 | Helped |
| freeze 7 rather than 8 bottom layers with early stopping (ablation) | Biography continual learning · Biography Task 0 | ACC: 42.46 -> 44.22 | Helped |
| freeze bottom 3 layers after the first task rather than 2 (ablation) | Continual instruction tuning · Continual Instruction Tuning (CIT, TRACE) | Test Score: 48.78 -> 50.33 | Helped |
| freeze bottom 3 rather than 2 layers after the first task (ablation) | Continual knowledge editing · Continual Knowledge Editing (CKE, ZSRE) | Efficacy: 70.65 -> 56.31 | Hurt |
Surprising findings
- Recovery experiments show Task 0 knowledge is nearly fully retained (about 96-100% recovered accuracy) even when first-token accuracy collapses to about 10%.
- Freezing bottom layers is a near-free improvement in continual learning, while regularization, generative replay, model merging and gradient projection methods perform poorly.
Limitations
- Main controlled experiments use a synthetic dataset and a randomly initialized model trained from scratch.
- Freeze still has a significant performance gap relative to data replay.
- Freezing bottom layers trades off stability and plasticity; freezing more layers slows learning of new tasks and causes late-stage forgetting.
Experimental setup
Datasets: Biography (synthetic) TRACE AOA alignment identity shifting instances ZSRE Concept-1K AdvBench
Benchmarks: Biography AdvBench TRACE ZSRE Concept-1K
Metrics: exact match accuracy first-token accuracy jailbreak rate test score efficacy paraphrase memorization accuracy generalization accuracy
Baselines: SEQ EWC LAMOL Task Vector Gradient Projection REPLAY
Builds on: EWC LAMOL Gradient Projection Task Vector
Closest papers by meaning
- Learn more, but bother less: parameter efficient continual learning (NeurIPS 2024 · similarity 0.91)
- Scalable Language Model with Generalized Continual Learning (ICLR 2024 · similarity 0.90)
- Geometry Conflict: Explaining and Controlling Forgetting in LLM Continual Post-Training (NeurIPS 2026 · similarity 0.89)
- Train-Attention: Meta-Learning Where to Focus in Continual Knowledge Learning (NeurIPS 2024 · similarity 0.89)
- Demystifying Language Model Forgetting with Low-rank Example Associations (NeurIPS 2025 · similarity 0.89)
- Unlocking the Power of Function Vectors for Characterizing and Mitigating Catastrophic Forgetting in Continual Instruction Tuning (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.