Research map / Label-efficient and robust learning
Self-supervised and contrastive: research map
988 accepted papers on Self-supervised and contrastive in Label-efficient and robust learning, from ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026), grouped into 3 clusters and 12 approaches. The busiest year so far is 2023.
Within Label-efficient and robust learning, its share held steady from 11.3% in 2023–24 to 10.5% in 2025–26 (324 → 321 papers at ICML, NeurIPS, CVPR and AAAI, the venues with data for all four years).
Explore Self-supervised and contrastive in the interactive map
Working on something in this topic? Describe your idea in scime atlas to see which approach it falls under, the closest papers by meaning and how crowded the spot has become.
Approaches and key papers
depth · self supervision · pre · 411 papers
Approaches in this cluster:
- Pretext-task visual pre-training (136 papers)
Pretrains with proxy tasks like reconstruction, colorization and self-supervised flow. - Self-supervision in supervised pipelines (121 papers)
Adds self-supervised objectives to supervised classifiers and other problem domains. - Self-supervised neuroimaging representations (98 papers)
Learns representations of brain signals and imaging with self-supervised objectives. - Self-supervised speech and brain decoding (56 papers)
Uses wav2vec-style speech models to model and decode brain activity.
Most cited and most cited since 2024:
- The Unreasonable Effectiveness of Deep Features as a Perceptual Metric (CVPR 2018 · 13,855 citations)
- TabNet: Attentive Interpretable Tabular Learning (AAAI 2021 · 1,769 citations)
- VoCo: A Simple-yet-Effective Volume Contrastive Learning Framework for 3D Medical Image Analysis (CVPR 2024 · 87 citations)
- Transfer CLIP for Generalizable Image Denoising (CVPR 2024 · 48 citations)
supervised ssl · representations · masked · 362 papers
Approaches in this cluster:
- Invariance-based self-supervised learning (168 papers)
Learns invariant representations through pretext invariance and causal or functional regularization. - Data-efficient and robust self-supervision (122 papers)
Improves self-supervised training with data selection, geometric harmonization and defenses against attacks. - Masked autoencoder pretraining (48 papers)
Improves masked autoencoders using mean teachers, curricula and frequency-aware or compressed designs. - Self-supervised speech representations (24 papers)
Learns speech and audio representations with self-supervised unified frameworks and pruning.
Most cited and most cited since 2024:
- Bootstrap Your Own Latent - A New Approach to Self-Supervised Learning (NeurIPS 2020 · 3,384 citations)
- Self-Supervised Learning of Pretext-Invariant Representations (CVPR 2020 · 1,405 citations)
- Transcriptomics-guided Slide Representation Learning in Computational Pathology (CVPR 2024 · 46 citations)
- SwitchTab: Switched Autoencoders Are Effective Tabular Learners (AAAI 2024 · 30 citations)
supervised contrastive · contrastive self · negative · 215 papers
Approaches in this cluster:
- Contrastive versus non-contrastive theory (74 papers)
Analyzes dynamics and duality of contrastive and non-contrastive self-supervised learning. - Inductive bias and minimal representations (66 papers)
Examines inductive bias and information content of contrastive representations and proposes variants. - Empirical contrastive visual learning (38 papers)
Frameworks and studies of when contrastive visual and dense representation learning works. - Representation evaluation and trade-offs (37 papers)
Evaluates universality, label efficiency and discriminative structure of learned representations.
Most cited and most cited since 2024:
- Momentum Contrast for Unsupervised Visual Representation Learning (CVPR 2020 · 12,585 citations)
- A Simple Framework for Contrastive Learning of Visual Representations (ICML 2020 · 7,181 citations)
- ODCR: Orthogonal Decoupling Contrastive Regularization for Unpaired Image Dehazing (CVPR 2024 · 31 citations)
- SCD-Net: Spatiotemporal Clues Disentanglement Network for Self-Supervised Skeleton-Based Action Recognition (AAAI 2024 · 24 citations)
Related topics in Label-efficient and robust learning
- Noisy labels (462)
- Zero-/few-shot and meta-learning (784)
- Test-time adaptation (359)
- OOD and anomaly detection (614)
- Recognition and long-tail (4,953)
- Semi-supervised learning (623)
- Domain adaptation and generalization (1,330)
- Knowledge distillation (544)