Research map / Vision-language and multimodal / CLIP and zero-shot
RLAP-CLIP: Continual Multimodal Learning with Prototype Adaptation and Difficulty-Aware Routing
Ruikun Luo, Jiarui Wang, Yuan Gao, Jing Yang, Jieming Yang, Song Wu, Hai Jin, Xiaoyu Xia
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add visual prompts to text-only prompting (ablation) | PROOF base · Eight datasets average | average accuracy (%): 78.97 -> 79.64 (+0.67) | Improved |
| add difficulty-aware MoE routing (ablation) | RLAP-CLIP with VP · Eight datasets average | average accuracy (%): 79.64 -> 80.79 (+1.15) | Improved |
| add RLPO prototype optimization (ablation) | RLAP-CLIP with VP+MoE · Eight datasets average | average accuracy (%): 80.79 -> 81.98 (+1.19) | Improved |
| add cross-modal attention fusion (CMF) (ablation) | RLAP-CLIP with VP+MoE+RLPO · Eight datasets average | average accuracy (%): 81.98 -> 82.87 (+0.89) | Improved |
| add visual prompts on Food-101 versus Aircraft (ablation) | PROOF base · Food-101 and Aircraft | average accuracy (%): Aircraft +1.24, Food-101 +0.16 | Mixed |
| add dual-modal prompting versus no prompting (ablation) | CLIP prompting · CUB-200 | separation ratio: 1.90 -> 3.11 | Improved |
| use dual-modal rather than text-only prompting (ablation) | CLIP prompting · CUB-200 | separation ratio: 2.74 -> 3.11 | Improved |
| replace simple averaging with learned RLPO weights (ablation) | prototype construction · Aircraft | accuracy (%): 56.32 -> 67.85 | Improved |
| replace simple averaging with RLPO weights (ablation) | prototype construction · ObjectNet | accuracy (%): 41.27% simple averaging | Improved |
| add quality-weighted averaging and then RLPO on CUB-200 (ablation) | prototype construction · CUB-200 | accuracy (%): 79.86 -> 83.42 | Improved |
| increase lambda_RLPO from 0.2 to 1.0 (ablation) | RLAP-CLIP · UCF-101 | accuracy (%): 97.68 -> 92.15 | Hurt |
| increase difficulty weight beta to 1.0 (ablation) | RLAP-CLIP · CUB-200 and Aircraft | accuracy (%): CUB-200 -4.44 points, Aircraft -4.58 points | Hurt |
Closest papers by meaning
- Test-Time Adaptation with CLIP Reward for Zero-Shot Generalization in Vision-Language Models (ICLR 2024 · similarity 0.87)
- CATP: Contextually Adaptive Token Pruning for Efficient and Enhanced Multimodal In-Context Learning (AAAI 2026 · similarity 0.87)
- CLIPTTA: Robust Contrastive Vision-Language Test-Time Adaptation (NeurIPS 2025 · similarity 0.87)
- A Hard-to-Beat Baseline for Training-free CLIP-based Adaptation (ICLR 2024 · similarity 0.87)
- Synergy and Diversity in CLIP: Enhancing Performance Through Adaptive Backbone Ensembling (ICLR 2025 · similarity 0.87)
- Post-pre-training for Modality Alignment in Vision-Language Foundation Models (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
