Research map / Continual learning / Continual fine-tuning of LLMs
Interactive Continual Learning: Fast and Slow Thinking
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Interactive Continual Learning (ICL), pairing a ViT System1 with a multimodal LLM System2 via CKT-MHA memory, CL-vMF EM-based memory optimization, and vMF-ODI hard-example routing to reduce catastrophic forgetting.
Problem
Catastrophic forgetting in continual learning, especially in open-class and class-incremental settings, where existing single-backbone rehearsal, regularization, and prompt-based methods degrade as tasks accumulate.
Approach
Inspired by Complementary Learning Systems, a fast ViT with a decoupled query/value class-task memory (trained with vMF posteriors and EM updates) handles most inputs, while a multimodal LLM is invoked only on batch-wise outlier hard examples, receiving the ViT's top-K predicted labels as prompt context.
Key results
- On ImageNet-R (10 splits, buffer 600), ICL with Pure-MM as System2 reaches 58.18% Class-IL and 82.64% Task-IL, versus 43.77% and 61.24% for DualPrompt with buffer.
- ICL without System2 reaches 94.60% Class-IL on CIFAR10 with no buffer, compared to 94.43% for DualPrompt without buffer.
- Adding the System2 module improves Class-IL accuracy by roughly 1-4 points across datasets and settings (e.g., CIFAR100 buffer 500: 80.67% to 82.43%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add System2 collaborative inference with Pure-MM (ablation) | ICL · CIFAR10 Class-IL | accuracy: 95.54 -> 96.83 | Helped |
| add System2 collaborative inference with Pure-MM (ablation) | ICL · CIFAR100 Class-IL | accuracy: 80.67 -> 82.43 | Helped |
| add System2 collaborative inference with Pure-MM (ablation) | ICL · ImageNet-R Class-IL | accuracy: 54.65 -> 58.18 | Helped |
| add System2 collaborative inference with Pure-MM (ablation) | ICL · CIFAR10 Task-IL | accuracy: 99.52 -> 99.68 | Helped |
| add System2 collaborative inference with Pure-MM (ablation) | ICL · CIFAR100 Task-IL | accuracy: 95.24 -> 96.35 | Helped |
| add System2 collaborative inference with Pure-MM (ablation) | ICL · ImageNet-R Task-IL | accuracy: 76.02 -> 82.64 | Helped |
| remove the EM strategy for memory training (ablation) | ICL CL-vMF · continual learning memory representation | memory separation | Hurt |
| increase the number of category choices K in the System2 prompt (ablation) | ICL · CIFAR10 Class-IL, CIFAR100 Class-IL, and ImageNet-R Class-IL | accuracy | Mixed |
| add margin gradient-stabilization regularization (ablation) | ICL CL-vMF · CIFAR100 and ImageNet-R | accuracy | Helped |
Surprising findings
- Adding a buffer to L2P and DualPrompt lowered their accuracy, which the authors attribute to interference from replayed samples.
Limitations
- Gains from System2 are modest (roughly 1-4 points) and depend on the choice of multimodal LLM and top-K prompt size.
- Performance is sensitive to the concentration parameter kappa on CIFAR100 and ImageNet-R.
- Only the System1 ViT is updated; System2 is kept frozen, and the LoRA fine-tuning option is described but not evaluated.
Experimental setup
Datasets: CIFAR10 CIFAR100 ImageNet-R
Benchmarks: CIFAR10 CIFAR100 ImageNet-R
Metrics: average incremental accuracy
Baselines: ER A-GEM iCaRL CVT SCoMMER BiMeCo DualNet L2P DualPrompt EWC LwF SI
Builds on: L2P DualPrompt DualNet EM algorithm LoRA
Closest papers by meaning
- Learn more, but bother less: parameter efficient continual learning (NeurIPS 2024 · similarity 0.88)
- Scalable Language Model with Generalized Continual Learning (ICLR 2024 · similarity 0.88)
- Improving Plasticity in Online Continual Learning via Collaborative Learning (CVPR 2024 · similarity 0.88)
- CMT: A Memory Compression Method for Continual Knowledge Learning of Large Language Models (AAAI 2025 · similarity 0.87)
- A Unified and General Framework for Continual Learning (ICLR 2024 · similarity 0.87)
- Train-Attention: Meta-Learning Where to Focus in Continual Knowledge Learning (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.