Research map / Continual learning / Continual fine-tuning of LLMs
Large Continual Instruction Assistant
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a continual instruction tuning framework that dynamically sets the EMA weight via a Taylor-expansion/Lagrange-multiplier derivation and groups instructions by TF-IDF similarity to retrain or expand LoRA parameters, evaluated on CoIN-style MLLM and T5 CIT benchmarks.
Problem
Continual instruction tuning of large foundation models suffers catastrophic forgetting under gradient updates, while a fixed EMA weight cannot balance plasticity and stability across evolving instruction datasets.
Approach
Derives a dynamic EMA weight beta_t from a Taylor expansion of the loss and Lagrange-multiplier optimization of stability/plasticity ideal conditions, approximates it layer-wise using gradients, and pairs it with a TF-IDF instruction codebook that decides whether to retrain or expand LoRA parameter groups.
Key results
- On LLaVA-7B, the method reaches 64.64 Avg.ACC and 1.93 Forgetting, versus 60.79 Avg.ACC and 5.42 Forgetting for the best baseline EProj.
- On LLaVA-13B, Avg.ACC is 65.10 and Forgetting 2.31, improving Avg.ACC by 3.68 over EProj.
- Ablation on LLaVA-7B: LoRA baseline 28.74 Avg.ACC and 37.29 Forgetting, stable EMA (beta=0.99) 48.09, dynamical EMA 55.33, and full method with instruction grouping 64.64.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use the proposed method instead of EProj | LLaVA-7B · CoIN benchmark | Avg.ACC: +3.85@Avg.ACC | Helped |
| use the proposed method instead of EProj | LLaVA-7B · CoIN benchmark | Forgetting: -3.49@Forgetting | Helped |
| use the proposed method instead of EProj | LLaVA-13B · CoIN benchmark | Avg.ACC: +3.68 | Helped |
| use the proposed method instead of EProj | LLaVA-13B · CoIN benchmark | Forgetting: -3.53 | Helped |
| replace stable EMA (β=0.99) with dynamical EMA (ablation) | CoIN continual tuning on LLaVA-7B · CoIN benchmark | Avg.ACC: +7.24@Avg.ACC | Helped |
| replace stable EMA (β=0.99) with dynamical EMA (ablation) | CoIN continual tuning on LLaVA-7B · CoIN benchmark | Forgetting: -9.20@Forgetting | Helped |
| add instruction grouping to dynamical EMA (ablation) | CoIN continual tuning on LLaVA-7B · CoIN benchmark | Avg.ACC: +9.31@Avg.ACC | Helped |
| add instruction grouping to dynamical EMA (ablation) | CoIN continual tuning on LLaVA-7B · CoIN benchmark | Forgetting: -5.11@Forgetting | Helped |
| add stable EMA (β=0.99) to LoRA baseline (ablation) | CoIN continual tuning on LLaVA-7B · CoIN benchmark | Avg.ACC: 28.74 -> 48.09 | Helped |
| add stable EMA (β=0.99) to LoRA baseline (ablation) | CoIN continual tuning on LLaVA-7B · CoIN benchmark | Forgetting: 37.29 -> 16.24 | Helped |
Surprising findings
- The method appears to suppress hallucinations during continual tuning, which the authors link to reduced forgetting.
- Fixed EMA weights of 0.990, 0.991, and 0.992 give noticeably different Avg.ACC (60.76, 58.83, 60.69), showing sensitivity to the summation weight.
Limitations
- Experiments are limited to image-text modalities due to computational constraints.
- The beta_t approximation relies on a heuristic: it falls back to a constant 0.99 when the computed value leaves (0,1), and the L1-norm simplification is justified only by empirical observation.
- Evaluation uses strict string-matching accuracy, which the authors note makes judgments more stringent and may not be comparable to other protocols.
Experimental setup
Datasets: ScienceQA TextVQA ImageNet GQA VizWiz Grounding VQAv2 OCR-VQA InstrDialog
Benchmarks: CoIN benchmark (ScienceQA, TextVQA, ImageNet, GQA, VizWiz, Grounding, VQAv2, OCR-VQA) InstrDialog
Metrics: Avg.ACC Forgetting New.ACC AR FWT BWT
Baselines: LoRA Fine-Tune MoELoRA LWF EWC MT PGP EProj Multi-Task
Builds on: exponential moving average LoRA EProj L2P CoIN
Closest papers by meaning
- CoIN: A Benchmark of Continual Instruction Tuning for Multimodel Large Language Models (NeurIPS 2024 · similarity 0.90)
- SAFE: Slow and Fast Parameter-Efficient Tuning for Continual Learning with Pre-Trained Models (NeurIPS 2024 · similarity 0.89)
- STAR: Stability-Inducing Weight Perturbation for Continual Learning (ICLR 2025 · similarity 0.89)
- SEFE: Superficial and Essential Forgetting Eliminator for Multimodal Continual Instruction Tuning (ICML 2025 · similarity 0.88)
- InfLoRA: Interference-Free Low-Rank Adaptation for Continual Learning (CVPR 2024 · similarity 0.87)
- Train with Perturbation, Infer after Merging: A Two-Stage Framework for Continual Learning (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.