Research map / Vision-language and multimodal / Cross-modal learning
CaPCL: Caption-Preserved Continual Learning for Text-to-Image Retrieval
Naoya Sogi, Ren Ohkubo, Takashi Shibata, Makoto Terao, Yusuke Hosoya, Takayuki Okatani
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add caption-based regularization (Lcap) | BLIP-2 continual T2I · avg over 4 orders | FM: 0.1241 (FT) -> 0.0654 | Improved |
| caption regularization vs similarity-matrix distillation | BLIP-2 · avg over 4 orders | AA: 0.6387 (ModX) -> 0.6942 | Improved |
| caption regularization in hardest order | BLIP-2 · PTRBI | FM: 0.2674 (FT) -> 0.0965 | Improved |
| EWC over plain fine-tuning | FT · avg over 4 orders | FM: 0.1241 -> 0.1241 | No clear effect |
| similarity-matrix distillation (DKR, C2MR) over EWC | BLIP-2 · avg over 4 orders | AA: 0.6460 -> 0.6300 | Hurt |
| use only Lacs (similarity distillation on auxiliary captions) (ablation) | CaPCL · avg over 4 orders | HM: 0.7966 -> 0.7272 | Hurt |
| use only Lacc instead of Lcap (ablation) | CaPCL · avg over 4 orders | HM: 0.7966 -> 0.7684 | Hurt |
| add Lacc and Lacs to Lcap (ablation) | CaPCL · avg over 4 orders | HM: 0.7966 -> 0.7987 | Improved |
| use Qwen2.5-VL captions instead of self-generated (ablation) | CaPCL · avg over 4 orders | HM: 0.7966 -> 0.7884 | Hurt |
| combine self and Qwen captions (ablation) | CaPCL · avg over 4 orders | HM: 0.7966 -> 0.8001 | Improved |
| apply auxiliary-caption contrastive loss to SigLIP2 | SigLIP2 · avg over 4 orders | HM: 0.6377 (FT) -> 0.7129 | Improved |
| high image overlap with high query divergence | BLIP-2 fine-tuning · Flickr30K MRR | forgetting: rho 0.33/0.45 -> 0.59 for joint score | Hurt |
Closest papers by meaning
- TIKP: Text-to-Image Knowledge Preservation for Continual Semantic Segmentation (AAAI 2024 · similarity 0.88)
- Enhancing Visual Continual Learning with Language-Guided Supervision (CVPR 2024 · similarity 0.87)
- Embracing Language Inclusivity and Diversity in CLIP through Continual Language Learning (AAAI 2024 · similarity 0.87)
- Continual Vision-Language Retrieval via Dynamic Knowledge Rectification (AAAI 2024 · similarity 0.86)
- Captured by Captions: On Memorization and its Mitigation in CLIP Models (ICLR 2025 · similarity 0.86)
- Visual Attention Never Fades: Selective Progressive Attention ReCalibration for Detailed Image Captioning in Multimodal Large Language Models (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
