Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning
Robust fine-tuning of zero-shot models
Improves robustness and prompt ensembles of zero-shot models, with theory for zero-shot prediction.
92 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2017–2026) · an approach in Zero-/few-shot and meta-learning, keyword cluster “zero shot · zsl · seen”
State History Benchmarks Tried Checklist Preprints Who Export
Where it stands
Quiet None of the tracked benchmarks has a report from 2025–26.
37 papers in 2023–24 and 26 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 18.0% to 14.8%.
No longer reported by papers here: ImageNet (accuracy, last in 2024).
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2017 · Zero-Shot Learning - the Good, the Bad and the Ugly (CVPR 2017 · 841 citations)
- 2018 · A Generative Adversarial Approach for Zero-Shot Learning From Noisy Texts (CVPR 2018 · 438 citations)
- 2019 · AutoVC: Zero-Shot Voice Style Transfer with Only Autoencoder Loss (ICML 2019 · 255 citations)
- 2020 · Learning Meta Model for Zero- and Few-Shot Face Anti-Spoofing (AAAI 2020 · 102 citations)
- 2021 · How Linguistically Fair Are Multilingual Pre-Trained Language Models? (AAAI 2021 · 41 citations)
- 2022 · Multitask Prompted Training Enables Zero-Shot Task Generalization (ICLR 2022 · 556 citations)
- 2023 · Finetune Like You Pretrain: Improved Finetuning of Zero-Shot Vision Models (CVPR 2023 · 90 citations)
- 2024 · One-Prompt to Segment All Medical Images (CVPR 2024 · 46 citations)
- 2025 · Logits DeConfusion with CLIP for Few-Shot Learning (CVPR 2025 · 18 citations)
- 2026 · Dual-Channel Learning Framework for Zero-Shot CircRNA-miRNA Interaction Prediction via State Space Modeling (AAAI 2026 · 8 citations)
Earliest papers here
- Zero-Shot Learning - the Good, the Bad and the Ugly (CVPR 2017 · 841 citations)
- A Generative Adversarial Approach for Zero-Shot Learning From Noisy Texts (CVPR 2018 · 438 citations)
- MSplit LBI: Realizing Feature Selection and Dense Estimation Simultaneously in Few-shot and Zero-shot Learning (ICML 2018 · 7 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
ImageNet · accuracy (higher is better)
4 papers, 2022–2024 · not reported since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2022 | 86.8 | Robust Fine-Tuning of Zero-Shot Models (CVPR 2022 · 400 citations) |
What has been tried
70 papers here report 749 measured changes (a component added, removed or swapped, compared with a baseline): helped 70%, no clear effect 6%, hurt 20%, mixed 5%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Use Stack MLP prompting with Gaussian Masking instead of the alternative Prompt-Parser combinations | held-out test set · Dice score | Helped | One-Prompt to Segment All Medical Images (CVPR 2024 · 46 citations) | |
| Increase the number of simulated peer students | Six DZCD target datasets · diagnostic performance | Mixed | Zero-1-to-3: Domain-Level Zero-Shot Cognitive Diagnosis via One Batch of Early-Bird Students towards Three Diagnostic Objectives (AAAI 2024 · 24 citations) | |
| Add MAF to the ZS-CLIP baseline | 11-dataset average, 16-shot · accuracy | 58.87 -> 70.83 | Helped | Logits DeConfusion with CLIP for Few-Shot Learning (CVPR 2025 · 18 citations) |
| Add HCA | ACDC adverse weather · Mean-mIoU | 39.67 -> 40.52 | Helped | Unified Language-driven Zero-shot Domain Adaptation (CVPR 2024 · 15 citations) |
| Add DTL without NCL | Cifar100, 16-shot · HCA | 10.04 -> 47.97 | Helped | ProTeCt: Prompt Tuning for Taxonomic Open Set Classification (CVPR 2024 · 5 citations) |
| add LMM instead of using DE/rand/1/bin mutation | BBOB d=30 · mean ranking score | 2.1667 -> 4.3333 | Helped | Pretrained Optimization Model for Zero-Shot Black Box Optimization (NeurIPS 2024 · 5 citations) |
| add negated feature generation to remove content information from the full audio representation | Seen and unseen speakers (averaged) · MCD | 7.44 -> 7.08 | Helped | Enhancing Zero-Shot Multi-Speaker TTS with Negated Speaker Representations (AAAI 2024 · 3 citations) |
| Replace batched OTTER with learned reweighting | CIFAR10 · accuracy (%) | 91.7 -> 88.4 | Hurt | OTTER: Effortless Label Distribution Adaptation of Zero-shot Models (NeurIPS 2024 · 3 citations) |
| Apply ZUT to ResNet-50 | VOC2007 · mAP | 90.29±0.15 -> 92.06±0.07 | Helped | Fair Training with Zero Inputs (AAAI 2025 · 2 citations) |
| Add the WRM loss to default-prompt ERM | iWildCam · macro F1 OOD | 41.9 -> 49.2 | Helped | Dual Risk Minimization: Towards Next-Level Robustness in Fine-tuning Zero-Shot Models (NeurIPS 2024 · 2 citations) |
| Use beam size 1 at both levels instead of beam size 5 | Italian-to-Dutch (Europarl) · BLEU | 26.10 -> 24.98 | Hurt | EBBS: An Ensemble with Bi-Level Beam Search for Zero-Shot Machine Translation (AAAI 2025 · 1 citation) |
| use adversarial pre-training instead of ordinary TabPFN pre-training | OpenML-CC18 small datasets · mean AUC | 0.913 ± 0.003 -> 0.921 ± 0.003 | Helped | Zero-shot Meta-learning for Tabular Prediction Tasks with Adversarially Pre-trained Transformer (ICML 2025 · 1 citation) |
What reviewers will expect
Datasets and baselines named in the 13 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- ImageNet · 3 of 13 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 63 papers; from OpenAlex and the papers' affiliations).
- Carnegie Mellon University · 6 papers
- Tsinghua University · 3 papers
- Xidian University · 3 papers
- Nanyang Technological University · 3 papers
- Hong Kong Baptist University · 2 papers
- University of Science and Technology of China · 2 papers
- Google · 2 papers
- University of California, San Diego · 2 papers
- Hong Kong University of Science and Technology · 2 papers
- Huawei · 2 papers
Use it
Download BibTeX of 39 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Generative feature synthesis for zero-shot (117 papers)
- Zero-shot robustness and transfer (101 papers)
- Transductive and semantic-dictionary zero-shot (90 papers)
Elsewhere in the topic
- Embedding adaptation for few-shot classification (71 papers)
- Fine-tuning strategies for few-shot (67 papers)
- Meta-augmentation and meta-objectives (60 papers)
- Fine-grained and cross-domain few-shot baselines (59 papers)
- Meta-learning analysis and benchmarks (50 papers)
- Normalization and unsupervised meta-learning (39 papers)
- Alignment-based few-shot metric learning (38 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).