Research map / Vision-language and multimodal / CLIP and zero-shot
Learning from Itself: Mining Internal Knowledge from Vision Language Models for Continual Learning
Yizheng Gong, Siyue Yu, Waleed Al-Nuaimy, Jimin Xiao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add auxiliary CLIP loss with generated tokens to CE (ablation) | CLIP finetuning · CIFAR-100 10-task | last accuracy: 80.91 -> 82.73 | Improved |
| add auxiliary CLIP loss with generated tokens to CE (ablation) | CLIP finetuning · CUB-200 10-task | last accuracy: 71.06 -> 74.12 | Improved |
| use auxiliary CLIP loss alone without classification loss (ablation) | CLIP · CIFAR-100 10-task | last accuracy: 71.11 -> 75.83 | No clear effect |
| add mutual distillation between vision and CLIP heads (ablation) | CE + vision CE · CUB-200 10-task | last accuracy: 71.48 -> 79.98 | Improved |
| add vision classifier head without distillation (ablation) | CLIP finetuning · CIFAR-100 10-task | last accuracy: 80.91 -> 81.22 | No clear effect |
| use symmetric mutual learning instead of adaptive distillation (ablation) | LfI · CIFAR-100 10-task | last accuracy: 82.73 -> 82.49 | No clear effect |
| use adaptive bidirectional distillation over unidirectional (ablation) | LfI · CUB-200 10-task | last accuracy: 80.82 -> 81.74 | Improved |
| allow vision head to train during unidirectional distillation (ablation) | LfI · CUB-200 10-task | last accuracy: 76.51 -> 80.82 | Improved |
| initialize generated tokens with class name tokens (ablation) | LfI token generation · CIFAR-100 10-task | last accuracy: 82.35 -> 83.82 | Improved |
| add COCO reference data in token generation (ablation) | LfI token generation · CIFAR-100 10-task | last accuracy: 82.77 -> 83.82 | Improved |
| use generated tokens instead of external MLLM captions | LfI · CIFAR-100 / CUB-200 | last accuracy: 83.82 vs 84.08; 81.74 vs 82.12 | No clear effect |
| use CLIP inference instead of vision-only inference (ablation) | LfI · CIFAR-100 10-task | last accuracy: 82.40 -> 83.82 | Improved |
Closest papers by meaning
- COMET: Decoupled Distillation, Routing, and Capacity Control for Task-Agnostic Continual Vision--Language Learning (NeurIPS 2026 · similarity 0.90)
- Interpreting and Analysing CLIP's Zero-Shot Image Classification via Mutual Knowledge (NeurIPS 2024 · similarity 0.89)
- Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapters (CVPR 2024 · similarity 0.89)
- Building Vision-Language Models on Solid Foundations with Masked Distillation (CVPR 2024 · similarity 0.88)
- Continual Vision-Language Retrieval via Dynamic Knowledge Rectification (AAAI 2024 · similarity 0.88)
- Hierarchical Semantic Tree Anchoring for CLIP-Based Class-Incremental Learning (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
