Research map / Label-efficient and robust learning
Self-supervised and contrastive: research map
1,267 papers on Self-supervised and contrastive in Label-efficient and robust learning, from ICML, NeurIPS, ICLR, CVPR, AAAI, ICCV and ECCV, and the journals TPAMI, IJCV and JMLR (2016–2026), grouped into 3 clusters and 12 approaches. The busiest year so far is 2023.
What it is
Self-supervised learning learns useful features from unlabeled data by solving tasks the data poses itself, such as recognising two crops of the same image (contrastive learning) or filling in masked patches. The features are then reused for classification, detection and more with few labels.
Terms you will see on this page:
- Contrastive learning
- pulling together representations of two views of the same sample and pushing apart others.
- Masked autoencoder (MAE)
- a model trained to reconstruct hidden patches of its input.
- Linear probe
- judging learned features by training only a linear classifier on top of them.
Start reading with the most cited papers here:
- The Unreasonable Effectiveness of Deep Features as a Perceptual Metric (CVPR 2018 · 13,855 citations)
- Momentum Contrast for Unsupervised Visual Representation Learning (CVPR 2020 · 12,585 citations)
Within Label-efficient and robust learning, its share held steady from 11.3% in 2023–24 to 10.5% in 2025–26 (324 → 321 papers at ICML, NeurIPS, ICLR, CVPR and AAAI, the venues with data for all four years).
Explore Self-supervised and contrastive in the interactive map
Working on something in this topic? Describe your idea in scime atlas to see which approach it falls under, the closest papers by meaning and how crowded the spot has become.
What changed recently
Approaches whose share of the topic moved most between 2023–24 and 2025–26 (ICML, NeurIPS, ICLR, CVPR and AAAI):
- Self-supervised neuroimaging representations rising: 7% → 17% of the topic (23 → 55 papers)
- Invariance-based self-supervised learning rising: 16% → 20% of the topic (53 → 65 papers)
- Self-supervised speech and brain decoding rising: 4% → 5% of the topic (12 → 17 papers)
- Contrastive versus non-contrastive theory falling: 9% → 5% of the topic (29 → 15 papers)
- Representation evaluation and trade-offs falling: 5% → 2% of the topic (16 → 5 papers)
Papers from 2025–2026 that are already cited most:
- AnySat: One Earth Observation Model for Many Resolutions, Scales, and Modalities (CVPR 2025 · 28 citations)
- Unsupervised Foundation Model-Agnostic Slide-Level Representation Learning (CVPR 2025 · 12 citations)
- Self-supervised Trusted Contrastive Multi-view Clustering with Uncertainty Refined (AAAI 2025 · 9 citations)
- mRNA2vec: mRNA Embedding with Language Model in the 5'UTR-CDS for mRNA Design (AAAI 2025 · 6 citations)
Approaches and key papers
Masked and invariance-based self-supervision · 521 papers
Keywords: supervised ssl, representations, masked
Approaches in this cluster:
- Invariance-based self-supervised learning (258 papers)
Learns invariant representations through pretext invariance and causal or functional regularization. - Data-efficient and robust self-supervision (164 papers)
Improves self-supervised training with data selection, geometric harmonization and defenses against attacks. - Masked autoencoder pretraining (74 papers)
Improves masked autoencoders using mean teachers, curricula and frequency-aware or compressed designs. - Self-supervised speech representations (25 papers)
Learns speech and audio representations with self-supervised unified frameworks and pruning.
Most cited and most cited since 2024:
- Bootstrap Your Own Latent - A New Approach to Self-Supervised Learning (NeurIPS 2020 · 3,384 citations)
- Self-Supervised Learning of Pretext-Invariant Representations (CVPR 2020 · 1,405 citations)
- Transcriptomics-guided Slide Representation Learning in Computational Pathology (CVPR 2024 · 46 citations)
- SwitchTab: Switched Autoencoders Are Effective Tabular Learners (AAAI 2024 · 30 citations)
Pretext tasks and self-supervision in practice · 474 papers
Keywords: depth, self supervision, pre
Approaches in this cluster:
- Pretext-task visual pre-training (173 papers)
Pretrains with proxy tasks like reconstruction, colorization and self-supervised flow. - Self-supervision in supervised pipelines (126 papers)
Adds self-supervised objectives to supervised classifiers and other problem domains. - Self-supervised neuroimaging representations (117 papers)
Learns representations of brain signals and imaging with self-supervised objectives. - Self-supervised speech and brain decoding (58 papers)
Uses wav2vec-style speech models to model and decode brain activity.
Most cited and most cited since 2024:
- The Unreasonable Effectiveness of Deep Features as a Perceptual Metric (CVPR 2018 · 13,855 citations)
- TabNet: Attentive Interpretable Tabular Learning (AAAI 2021 · 1,769 citations)
- VoCo: A Simple-yet-Effective Volume Contrastive Learning Framework for 3D Medical Image Analysis (CVPR 2024 · 87 citations)
- Transfer CLIP for Generalizable Image Denoising (CVPR 2024 · 48 citations)
Contrastive learning · 272 papers
Keywords: supervised contrastive, contrastive self, negative
Approaches in this cluster:
- Contrastive versus non-contrastive theory (98 papers)
Analyzes dynamics and duality of contrastive and non-contrastive self-supervised learning. - Inductive bias and minimal representations (82 papers)
Examines inductive bias and information content of contrastive representations and proposes variants. - Empirical contrastive visual learning (52 papers)
Frameworks and studies of when contrastive visual and dense representation learning works. - Representation evaluation and trade-offs (40 papers)
Evaluates universality, label efficiency and discriminative structure of learned representations.
Most cited and most cited since 2024:
- Momentum Contrast for Unsupervised Visual Representation Learning (CVPR 2020 · 12,585 citations)
- A Simple Framework for Contrastive Learning of Visual Representations (ICML 2020 · 7,181 citations)
- ODCR: Orthogonal Decoupling Contrastive Regularization for Unpaired Image Dehazing (CVPR 2024 · 31 citations)
- SCD-Net: Spatiotemporal Clues Disentanglement Network for Self-Supervised Skeleton-Based Action Recognition (AAAI 2024 · 24 citations)
Related topics in Label-efficient and robust learning
- Noisy labels (576)
- Zero-/few-shot and meta-learning (1,029)
- Test-time adaptation (437)
- OOD and anomaly detection (836)
- Recognition and long-tail (6,306)
- Semi-supervised learning (897)
- Domain adaptation and generalization (2,012)
- Knowledge distillation (783)
