AI Paper Atlas

Research map / Vision-language and multimodal / CLIP and zero-shot

A Retrospect to Multi-prompt Learning across Vision and Language

Ziliang Chen, Xin Huang, Quanlong Guan, Liang Lin, Weiqi Luo

ICCV 2023 · Vision-language and multimodal › CLIP and zero-shot · approach: Prompt learning for CLIP · 11 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
generate prompts with EMPL energy-based sampler for CoOpCoOp · Base-to-new generalization (avg)New-class accuracy: 63.22 -> 70.93Improved
generate prompts with EMPL energy-based sampler for ProDAProDA · Base-to-new generalization (avg)Harmonic mean: 76.65 -> 77.39Improved
add EMPL to CoOp/ProDA per dataset in base-to-newCoOp and ProDA · 10 datasets base-to-newBase accuracy per dataset: 7 of 10 (CoOp), 6 of 10 (ProDA) improvedMixed
add EMPL to CoOp and ProDA in cross-domain generalizationCoOp and ProDA · ImageNet variantsaccuracy: improves 7 of 8 target settings; CoOp ImageNet 71.51 -> 70.89Mixed
add EMPL to CoOp in cross-dataset transferCoOp · Cross-dataset transferAvg target accuracy: 63.88 -> 66.49Improved
add EMPL to CoOp for few-shot image-text retrievalCoOp · MSCOCO retrieval (1% data)R@1: 53.58 -> 56.85 (+3.37)Improved
apply SGLD sampling through more (low-level) transformer layers (ablation)EMPL (+CoOp) · DTD base-to-newHarmonic mean: 63.32 -> 61.62 (2 vs 9 layers); 0.048 -> 0.278 sec/iterNo clear effect
increase lambda weight on open-vocabulary energy term (ablation)EMPL (+CoOp) · DTD base-to-newnew-class accuracyMixed
draw more prompt samples from the energy-based distribution (ablation)EMPL (+CoOp) · DTD base-to-newHarmonic meanImproved
add more learnable prompts (CLIP to CoOp to ProDA) to shrink modality gapprompt learning · MsCOCO modality gapgap magnitudeImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy