Research map / Continual learning / Online and replay-based
Unlocking the Power of Function Vectors for Characterizing and Mitigating Catastrophic Forgetting in Continual Instruction Tuning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Uses function vectors (FV) to characterize catastrophic forgetting in LLMs during continual instruction tuning, and proposes FV-guided training (an FV consistency loss plus FV-guided KL loss) that reduces forgetting when added to existing continual learning methods.
Problem
Catastrophic forgetting in LLMs during continual instruction tuning is poorly understood at the internal-mechanism level, and existing task-similarity measures (feature/readout similarity, parameter L2 distance) do not predict model-dependent forgetting well.
Approach
Interpret forgetting via function vectors (a compact causal-head summary of a task's function), finding forgetting stems from biased activation of the task latent variable rather than overwriting task functions; then regularize FV shifts and align zero-shot logits with FV-intervened logits during fine-tuning.
Key results
- Adding FVG to InsCL on Llama2-7b-chat improves IP by 8.30 on NI-Seq-G1 and 16.11 on NI-Seq-C1 while changing FP by under 1.5 points in most settings.
- Across methods on Llama2-7b-chat, FVG yields average gains of 5.44 in GP and 17.52 in IP.
- With IncLoRA on TRACE, FVG improves FP by 12.13 on Llama2-7b-chat; under InsCL on NI-Seq-M1 FP drops by 2.22 (reported as a drawback).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add function vector-guided training to LoraInc | Llama2-7B-chat with LoraInc · NI-Seq-G1 | IP: +25.25 | Helped |
| Add function vector-guided training to LoraInc | Llama2-7B-chat with LoraInc · NI-Seq-C1 | IP: +25.53 | Helped |
| Add function vector-guided training to LoraInc | Llama2-7B-chat with LoraInc · NI-Seq-M1 | IP: +15.78 | Helped |
| Add function vector-guided training to LoraInc | Llama2-7B-chat with LoraInc · TRACE | IP: +16.17 | Helped |
| Add function vector-guided training to EWC | Llama2-7B-chat with EWC · NI-Seq-G1 | IP: +27.18 | Helped |
| Add function vector-guided training to EWC | Llama2-7B-chat with EWC · TRACE | FP: -8.77 | Hurt |
| Add function vector-guided training to O-lora | Llama2-7B-chat with O-lora · NI-Seq-M1 | FP: -0.42 | Hurt |
| Add function vector-guided training to InsCL | Llama2-7B-chat with InsCL · NI-Seq-M1 | FP: -2.22 | Hurt |
| Add function vector-guided training to InsCL | Llama3-8B-chat with InsCL · NI-Seq-M1 | FP: -2.80 | Hurt |
| Add the source evaluation-task function vector during inference | Continually tuned LLM · HellaSwag | performance: 7.3 | Helped |
| Subtract the current training-task function vector during inference | Llama2-7B-chat at M5 · HellaSwag | performance: 39.5 to 49.1 | Helped |
| Train on a generation-task sequence rather than a classification-task sequence | Llama3-8B-chat · NI-Seq-G1 vs NI-Seq-C1 | Forget: 1.3 vs 7.2 | Hurt |
Surprising findings
- Forgetting often reverses: performance initially drops then rebounds over later training stages (e.g., Object Count rising to 122% after two stages).
- Models with low function-vector similarity between training and evaluation tasks showed more forgetting, contrary to the feature-similarity hypothesis from prior work.
Limitations
- FVG adds regularization terms requiring head selection (top-10 causal heads) and hyperparameter search (alpha grid search), and the layer for intervention is fixed to 9.
- FVG can reduce plasticity on some settings (e.g., FP drop of 2.22 with InsCL on NI-Seq-M1).
- Evaluation uses only four general tasks and a few sequences, truncated to 190 samples per dataset; code release was only promised.
Experimental setup
Datasets: SuperNI TRACE Hellaswag CommonsenseQA Alpaca BBH-Object-Count
Benchmarks: SuperNI sequences (NI-Seq-C1, NI-Seq-G1, NI-Seq-M1) TRACE Hellaswag CommonsenseQA Alpaca BBH-Object-Count
Metrics: GP (zero-shot general performance) IP (in-context general performance) FP (final trained-task performance) AP Forget Rouge-L
Baselines: IncLoRA EWC O-LoRA InsCL
Builds on: function vector Incremental LoRA Elastic Weight Consolidation O-LoRA InsCL
Closest papers by meaning
- Spurious Forgetting in Continual Learning of Language Models (ICLR 2025 · similarity 0.89)
- STAR: Stability-Inducing Weight Perturbation for Continual Learning (ICLR 2025 · similarity 0.89)
- Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks (ICML 2025 · similarity 0.89)
- The Joint Effect of Task Similarity and Overparameterization on Catastrophic Forgetting — An Analytical Model (ICLR 2024 · similarity 0.89)
- Unlocking the Power of Rehearsal in Continual Learning: A Theoretical Perspective (ICML 2025 · similarity 0.88)
- Predicting the Susceptibility of Examples to Catastrophic Forgetting (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.