Research map / Vision-language and multimodal / CLIP and zero-shot
CASPA: Graph-Structured Concept Anchors for Modality-Agnostic Adaptation in Vision-Language Models
Abhiroop Chatterjee, Susmita Ghosh, Ashish Ghosh, Emmett Ientilucci
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add text and image shift modules (ablation) | CASPA · 4-shot ImageNet/Flowers/DTD (ViT-B/16) | accuracy: 64.17 -> 70.28 (ImageNet) | Improved |
| add anchors (K=0 -> K=48) (ablation) | CASPA · DTD 4-shot | accuracy: 51.76 -> 65.43 | Improved |
| add cross-consistency regularization lambda_x (ablation) | CASPA · 4-shot ImageNet/Flowers/DTD (ViT-B/16) | accuracy: 64.36 -> 65.43 (DTD) | Improved |
| cosine similarity instead of MSE for consistency (ablation) | CASPA · 4-shot ImageNet/Flowers/DTD (ViT-B/16) | accuracy: 64.15 -> 65.43 (DTD) | Improved |
| add diversity regularizer lambda_d=0.05 (ablation) | CASPA · 4-shot ImageNet/Flowers/DTD (ViT-B/16) | accuracy: 65.11 -> 65.43 (DTD) | Improved |
| increase anchors from 48 to 64 (ablation) | CASPA · DTD 4-shot | accuracy: 65.43 -> 64.68 | Hurt |
| text shift only vs image shift only (ablation) | CASPA · ImageNet (ViT-B/32), 1-8 shot | accuracy: 62.40 -> 65.30 (4-shot, image-only vs text-only) | Improved |
| add ASAM optimizer (ablation) | CASPA · EuroSAT Base-to-Novel | Novel accuracy: 67.46% -> 82.06% | Improved |
| raise adaptive scaling amax from 1 to 5-7 (ablation) | CASPA · UCF Base-to-Novel | Novel accuracy: 23.32% at 1 -> full potential at 5-7 | Improved |
| add CASPA-G on top of CoOp (ablation) | CoOp · Base-to-Novel (11-dataset average) | HM: 71.66 -> 77.98 | Improved |
| add Gaussian-weighted aggregation initialization (CASPA-A) (ablation) | CASPA-G · Base-to-Novel (11-dataset average) | HM: 81.01 -> 81.27 | Improved |
| anchor-based adapter vs RAda | CASPA-G · Base-to-Novel DTD | HM: 73.86 -> 70.78 | Hurt |
Closest papers by meaning
- MMA: Multi-Modal Adapter for Vision-Language Models (CVPR 2024 · similarity 0.90)
- Exploring the Better Multimodal Synergy Strategy for Vision-Language Models (AAAI 2025 · similarity 0.89)
- Rethinking Misalignment in Vision-Language Model Adaptation from a Causal Perspective (NeurIPS 2024 · similarity 0.89)
- Concept-Guided Prompt Learning for Generalization in Vision-Language Models (AAAI 2024 · similarity 0.89)
- Domain-Agnostic Mutual Prompting for Unsupervised Domain Adaptation (CVPR 2024 · similarity 0.88)
- Understanding Retrieval-Augmented Task Adaptation for Vision-Language Models (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
