Research map / Continual learning / Continual fine-tuning of LLMs
COMET: Decoupled Distillation, Routing, and Capacity Control for Task-Agnostic Continual Vision--Language Learning
Seungyoon Woo, Keonhee Park, Gunhee Kim
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes COMET, a task-agnostic continual learning method for vision-language models (e.g., CLIP) that decouples distillation, routing, and capacity control to address modal-consistency drift and related pressures.
Problem
Task-agnostic continual learning for VLMs such as CLIP, where tasks arrive sequentially without past data or task identity and the backbone cannot be retrained, causing modal-consistency drift and representation-related interference beyond the usual stability-plasticity tradeoff.
Approach
Decouples three mechanisms (distillation, routing, and capacity control) to jointly handle the coupled pressures of task-agnostic continual vision-language learning; details beyond the abstract are not available.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Tri-Affinity Distillation (TAD) (ablation) | COMET base model · X-TAIL full-shot | Transfer: 56.6 -> 61.3 | Improved |
| add Student-Aware Cross-Expert Distillation (SACED) alone (ablation) | COMET base model · X-TAIL full-shot | Transfer: 56.6 -> 59.9 | Improved |
| add SACED alone (ablation) | COMET base model · X-TAIL full-shot | Last: 85.6 -> 85.3 | No clear effect |
| add SACED on top of TAD (ablation) | COMET · X-TAIL full-shot | Transfer / Average / Last: 61.3 -> 62.6 Transfer; 75.4 -> 76.1 Average; 87.6 -> 88.1 Last | Improved |
| add both TAD and SACED in 16-shot setting (ablation) | COMET · X-TAIL 16-shot | Last: 73.3 -> 83.3 | Improved |
| use bandit-controlled merging instead of Always-Expand (ablation) | COMET · X-TAIL full-shot | Transfer: 61.6 -> 62.6 with 10 -> 8 experts | Improved |
| use Always-Merge or random merging instead of CMAB (ablation) | COMET · X-TAIL full-shot | Last: 88.1 -> 58.4 (Always-Merge); 69.1 (Random) | Hurt |
| add SACED to reduce routing-induced interference (ablation) | COMET · X-TAIL misrouted samples | accuracy on misrouted samples: +10.8 pp; 52.5% of misrouted still correct | Improved |
| use full per-task experts with bandit merging instead of adapters | COMET · heterogeneous 4-task | Transfer: 12.5 -> 31.2 | Improved |
| use smaller TinyCLIP per-task experts (58M) instead of CLIP-B/16 (150M) | COMET · X-TAIL full-shot | Transfer / Average / Last: vs LADA: +1.5, +1.4, +1.4 pp | Improved |
| use reconstruction router with sparse top-1 path | COMET · inference cost | latency (ms): 176.9 -> 107.1 (vs LADA) | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- LADA — LADA: Scalable Label-Specific CLIP Adapter for Continual Learning (ICML 2025) · better on 6 of 6 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Memory-Free Continual Learning with Null Space Adaptation for Zero-Shot Vision-Language Models (ICLR 2026 · similarity 0.91)
- Beyond Disjoint Tasks: Towards More Natural Continual Learning for Vision-Language Models (ECCV 2026 · similarity 0.91)
- Enhanced Continual Learning of Vision-Language Models with Model Fusion (ICLR 2026 · similarity 0.91)
- Vision-language Incremental Learning with Dual Class-individual Memory (AAAI 2026 · similarity 0.90)
- Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapters (CVPR 2024 · similarity 0.89)
- Synthetic Data is an Elegant GIFT for Continual Vision-Language Models (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
