Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning
Zero-shot robustness and transfer
Studies and improves generalization of foundation models to new groups, domains and data.
101 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Zero-/few-shot and meta-learning, keyword cluster “zero shot · zsl · seen”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
31 papers in 2023–24 and 49 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic grew, from 15.0% to 27.8%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Learning Deep Parsimonious Representations (NeurIPS 2016 · 88 citations)
- 2017 · Automatic Discovery, Association Estimation and Learning of Semantic Attributes for a Thousand Categories (CVPR 2017 · 27 citations)
- 2018 · Generalizing and Improving Bilingual Word Embedding Mappings with a Multi-Step Framework of Linear Transformations (AAAI 2018 · 233 citations)
- 2019 · Dimensionality Reduction for Representing the Knowledge of Probabilistic Models (ICLR 2019 · 8 citations)
- 2020 · Multilingual Alignment of Contextual Word Representations (ICLR 2020 · 151 citations)
- 2021 · Proxy Synthesis: Learning with Synthetic Classes for Deep Metric Learning (AAAI 2021 · 34 citations)
- 2022 · BatchFormer: Learning To Explore Sample Relationships for Robust Representation Learning (CVPR 2022 · 95 citations)
- 2023 · Prompting Large Language Model for Machine Translation: A Case Study (ICML 2023 · 69 citations)
- 2024 · StyleSinger: Style Transfer for Out-of-Domain Singing Voice Synthesis (AAAI 2024 · 18 citations)
- 2025 · PertEval-scFM: Benchmarking Single-Cell Foundation Models for Perturbation Effect Prediction (ICML 2025 · 40 citations)
- 2026 · Relational Transformer: Toward Zero-Shot Foundation Models for Relational Data (ICLR 2026 · 1 citation)
Earliest papers here
- Learning Deep Parsimonious Representations (NeurIPS 2016 · 88 citations)
- Automatic Discovery, Association Estimation and Learning of Semantic Attributes for a Thousand Categories (CVPR 2017 · 27 citations)
- Learning Attributes from the Crowdsourced Relative Labels (AAAI 2017 · 9 citations)
What has been tried
65 papers here report 757 measured changes (a component added, removed or swapped, compared with a baseline): helped 64%, no clear effect 10%, hurt 21%, mixed 5%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Increase probe parameter count | Perturbation effect prediction · prediction performance | No clear effect | PertEval-scFM: Benchmarking Single-Cell Foundation Models for Perturbation Effect Prediction (ICML 2025 · 40 citations) | |
| Use the full three data sources (Dreal, Ddrand, Dflow) rather than Dreal alone | SMILE-UHURA zero-shot · Dice | 65.45 -> 74.66 | Helped | vesselFM: A Foundation Model for Universal 3D Blood Vessel Segmentation (CVPR 2025 · 23 citations) |
| add UMLN | OOD parallel style transfer · CMOS | -0.28 -> 0.00 | Helped | StyleSinger: Style Transfer for Out-of-Domain Singing Voice Synthesis (AAAI 2024 · 18 citations) |
| Add label-anchored contrastive learning to sentence-anchored contrastive learning | FewRel 1.0 validation, 5-way-1-shot · accuracy | 97.65 -> 98.17 | Helped | Synergistic Anchored Contrastive Pre-training for Few-Shot Relation Extraction (AAAI 2024 · 10 citations) |
| Randomly split the training data into two subsets instead of clustering | CLIP benchmark (400M image-caption pairs, ViT-B/32) · average accuracy | 58.6 -> 57.7 | Hurt | MoDE: CLIP Data Experts via Clustering (CVPR 2024 · 10 citations) |
| increase hallucinated features from 5k to 50k | LibriSpeech test-clean · WER | 6.14 -> 4.84 | Helped | Phoneme Hallucinator: One-Shot Voice Conversion via Set Expansion (AAAI 2024 · 7 citations) |
| add source and target prompt templates rather than using none | PDNC · accuracy | 56.34 -> 68.43 | Helped | SIG: Speaker Identification in Literature via Prompt-Based Generation (AAAI 2024 · 7 citations) |
| Combine country-name prompts with country-specific LLM descriptors rather than either knowledge source alone | DollarStreet · top-1 balanced accuracy | Mixed | Incorporating Geo-Diverse Knowledge into Prompting for Increased Geographical Robustness in Object Recognition (CVPR 2024 · 3 citations) | |
| Add tensor-norm loss to a single encoder | STS-B testset · Spearman correlation | 77.75 -> 78.37 | Helped | TNCSE: Tensor Norm Constraints for Unsupervised Contrastive Learning of Sentence Embeddings (AAAI 2025 · 2 citations) |
| Simplify captions to labels only | Six shifted ImageNet versions · OOD accuracy | Hurt | Understanding the Robustness of Multi-modal Contrastive Learning to Distribution Shift (ICLR 2024 · 2 citations) | |
| increase forget data proportion from 5% to 15% | CIFAR-10 · test accuracy | 73% -> 59% | Hurt | Towards Source-Free Machine Unlearning (CVPR 2025 · 1 citation) |
| add TRIE | Trajectory prediction, Chengdu · RMSE (meters) | 236.18 -> 197.91 | Helped | TransferTraj: A Vehicle Trajectory Learning Model for Region and Task Transferability (NeurIPS 2025 · 1 citation) |
What reviewers will expect
Datasets and baselines named in the 28 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- CIFAR-100 · 2 of 28 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 80 papers; from OpenAlex and the papers' affiliations).
- Fudan University · 4 papers
- Chinese Academy of Sciences · 3 papers
- Huawei · 3 papers
- Amazon · 3 papers
- Stanford University · 3 papers
- Tsinghua University · 3 papers
- Carnegie Mellon University · 3 papers
- Tencent · 2 papers
- Zhejiang University · 2 papers
- University of Pittsburgh · 2 papers
Use it
Download BibTeX of 37 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Generative feature synthesis for zero-shot (117 papers)
- Robust fine-tuning of zero-shot models (92 papers)
- Transductive and semantic-dictionary zero-shot (90 papers)
Elsewhere in the topic
- Embedding adaptation for few-shot classification (71 papers)
- Fine-tuning strategies for few-shot (67 papers)
- Meta-augmentation and meta-objectives (60 papers)
- Fine-grained and cross-domain few-shot baselines (59 papers)
- Meta-learning analysis and benchmarks (50 papers)
- Normalization and unsupervised meta-learning (39 papers)
- Alignment-based few-shot metric learning (38 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).