Research map / Vision-language and multimodal / CLIP and zero-shot
Parameter-efficient visual tuning
Adapts pretrained vision and multimodal models through visual prompt tuning or efficient fine-tuning.
41 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2022–2026) · an approach in CLIP and zero-shot, keyword cluster “prompt · tuning · pre trained”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
18 papers in 2023–24 and 22 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic held steady, from 4.9% to 4.4%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2022 · Expanding Large Pre-Trained Unimodal Models With Multimodal Information Injection for Image-Text Multimodal Classification (CVPR 2022 · 40 citations)
- 2023 · Visual Prompt Multi-Modal Tracking (CVPR 2023 · 347 citations)
- 2024 · DGL: Dynamic Global-Local Prompt Tuning for Text-Video Retrieval (AAAI 2024 · 34 citations)
- 2025 · Densely Connected Parameter-Efficient Tuning for Referring Image Segmentation (AAAI 2025 · 19 citations)
- 2026 · PEFT-BoA: Parameter-Efficient Fine-Tuning with Bag-of-Adapters for Multi-Modal Object Re-identification (AAAI 2026 · 1 citation)
Earliest papers here
- Expanding Large Pre-Trained Unimodal Models With Multimodal Information Injection for Image-Text Multimodal Classification (CVPR 2022 · 40 citations)
- Visual Prompt Multi-Modal Tracking (CVPR 2023 · 347 citations)
- POUF: Prompt-Oriented Unsupervised Fine-tuning for Large Pre-trained Models (ICML 2023 · 7 citations)
What has been tried
28 papers here report 339 measured changes (a component added, removed or swapped, compared with a baseline): helped 70%, no clear effect 4%, hurt 23%, mixed 3%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Add global and local video attention with shared parameters | MSR-VTT-9K, Text→Video · R@1 | 43.5 -> 45.8 | Helped | DGL: Dynamic Global-Local Prompt Tuning for Text-Video Retrieval (AAAI 2024 · 34 citations) |
| add query-residual adaptation to the standard Adapter | DiDeMo test, text-to-video retrieval · R@1 | 47.4 -> 48.7 | Helped | UniAdapter: Unified Parameter-Efficient Transfer Learning for Cross-modal Modeling (ICLR 2024 · 20 citations) |
| add Dense Aligner | RefCOCO · Avg IoU | 74.6 -> 75.0 | Helped | Densely Connected Parameter-Efficient Tuning for Referring Image Segmentation (AAAI 2025 · 19 citations) |
| Add disentanglement loss | CelebA Attractive · Equalized Odds | 15.0 -> 12.0 | Helped | Fair-VPT: Fair Visual Prompt Tuning for Image Classification (CVPR 2024 · 15 citations) |
| replace FFT with a fixed linear transform | VTAB-1k Natural and Specialized · accuracy | 81.35% -> 80.98%; 84.93% -> 84.02% | Hurt | Visual Fourier Prompt Tuning (NeurIPS 2024 · 15 citations) |
| Add dynamic positive centroid sampling (DPS) instead of RPS | Waterbirds · WGA | 73.08 -> 76.93 | Helped | Calibrating Multi-modal Representations: A Pursuit of Group Robustness without Annotations (CVPR 2024 · 14 citations) |
| Add LSkip to full fine-tuning | Base-to-new generalization, 11 datasets (Avg) · H | 79.27 -> 81.00 | Helped | Skip Tuning: Pre-trained Vision-Language Models are Effective and Efficient Adapters Themselves (CVPR 2025 · 8 citations) |
| add both metric learning losses LML(x, p) and LML(p, xcls) | VTAB-1K Natural · accuracy | +1.08 pp | Helped | DA-VPT: Semantic-Guided Visual Prompt Tuning for Vision Transformers (CVPR 2025 · 7 citations) |
| Add visual prompts and full-finetune the backbone | VTAB-1k Natural · accuracy | 75.19 -> 72.80 | Hurt | Facing the Elephant in the Room: Visual Prompt Tuning or Full finetuning? (ICLR 2024 · 6 citations) |
| use multi-branch retrieval rather than cross-modal retrieval | MicroLens · nMSE | 0.7441 -> 0.7338 | Helped | In-context Prompt-augmented Micro-video Popularity Prediction (AAAI 2025 · 5 citations) |
| Increase the C-TPT weight lambda | Multiple validation datasets · ECE | Helped | C-TPT: Calibrated Test-Time Prompt Tuning for Vision-Language Models via Text Feature Dispersion (ICLR 2024 · 5 citations) | |
| add gating balancing loss | ANLI-R1, ANLI-R2, ANLI-R3, QNLI, WNLI (average) · Average score | 77.57 -> 78.07 | Helped | Mixture of LoRA Experts (ICLR 2024 · 3 citations) |
What reviewers will expect
Datasets and baselines named in the 13 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- ImageNet · 5 of 13 papers
- StanfordCars · 5 of 13 papers
- Flowers102 · 4 of 13 papers
- DTD · 4 of 13 papers
- OxfordPets · 4 of 13 papers
- UCF101 · 4 of 13 papers
- Caltech101 · 4 of 13 papers
- Food101 · 4 of 13 papers
Baselines
- CoOp · 4 of 13 papers
- CoCoOp · 4 of 13 papers
- MaPLe · 4 of 13 papers
- LoRA · 3 of 13 papers
- PromptSRC · 3 of 13 papers
- DePT · 3 of 13 papers
- KgCoOp · 2 of 13 papers
- TCP · 2 of 13 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 40 papers; from OpenAlex and the papers' affiliations).
- Meta · 4 papers
- Zhejiang University · 3 papers
- Microsoft · 3 papers
- Google · 3 papers
- University of California, Los Angeles · 3 papers
- Tsinghua University · 2 papers
- University of Central Florida · 2 papers
- Rochester Institute of Technology · 2 papers
- Virginia Tech · 2 papers
- Huawei · 2 papers
Use it
Download BibTeX of 26 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Prompt learning for CLIP (102 papers)
- Test-time adaptation of VLMs (93 papers)
- Feature calibration for unseen-class detection (50 papers)
Elsewhere in the topic
- Open-vocabulary alignment from VLMs (130 papers)
- Attribute-based zero-shot learning (115 papers)
- Adapting VLMs for zero-shot classification (108 papers)
- Recaptioning and text-supervised pretraining (73 papers)
- CLIP training recipes and data curation (72 papers)
- Language models for zero-shot reasoning (64 papers)
- Fine-grained improvements to CLIP (63 papers)
- Self-distillation for vision-language pretraining (48 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).