Research map / Label-efficient and robust learning / Self-supervised and contrastive
Pretext-task visual pre-training
Pretrains with proxy tasks like reconstruction, colorization and self-supervised flow.
136 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Self-supervised and contrastive, keyword cluster “depth · self supervision · pre”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
45 papers in 2023–24 and 37 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 13.9% to 11.5%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Homography Estimation From the Common Self-Polar Triangle of Separate Ellipses (CVPR 2016 · 15 citations)
- 2017 · Colorization as a Proxy Task for Visual Understanding (CVPR 2017 · 487 citations)
- 2018 · Cross-Domain Self-Supervised Multi-Task Feature Learning Using Synthetic Imagery (CVPR 2018 · 200 citations)
- 2019 · SelFlow: Self-Supervised Learning of Optical Flow (CVPR 2019 · 339 citations)
- 2020 · Generative Pretraining From Pixels (ICML 2020 · 549 citations)
- 2021 · HR-Depth: High Resolution Self-Supervised Monocular Depth Estimation (AAAI 2021 · 300 citations)
- 2022 · Self-Supervised Pre-Training of Swin Transformers for 3D Medical Image Analysis (CVPR 2022 · 834 citations)
- 2023 · Beyond Appearance: A Semantic Controllable Self-Supervised Learning Framework for Human-Centric Visual Tasks (CVPR 2023 · 154 citations)
- 2024 · VoCo: A Simple-yet-Effective Volume Contrastive Learning Framework for 3D Medical Image Analysis (CVPR 2024 · 87 citations)
- 2025 · Decoupled Spatio-Temporal Consistency Learning for Self-Supervised Tracking (AAAI 2025 · 14 citations)
- 2026 · OlmoEarth: Stable Latent Image Modeling for Multimodal Earth Observation (CVPR 2026 · 2 citations)
Earliest papers here
- Homography Estimation From the Common Self-Polar Triangle of Separate Ellipses (CVPR 2016 · 15 citations)
- Identifying Good Training Data for Self-Supervised Free Space Estimation (CVPR 2016 · 7 citations)
- Colorization as a Proxy Task for Visual Understanding (CVPR 2017 · 487 citations)
What has been tried
119 papers here report 1,355 measured changes (a component added, removed or swapped, compared with a baseline): helped 72%, no clear effect 6%, hurt 19%, mixed 3%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| add the position-prediction loss Lpred | BTCV · Average Dice Score (%) | 80.53 -> 82.96 | Helped | VoCo: A Simple-yet-Effective Volume Contrastive Learning Framework for 3D Medical Image Analysis (CVPR 2024 · 87 citations) |
| add the noisy input to the decoder | Kodak24, Gaussian σ = 50 · PSNR | 21.76 -> 26.87 | Helped | Transfer CLIP for Generalizable Image Denoising (CVPR 2024 · 48 citations) |
| Add Fourier-domain reconstruction loss (α=0.01) | RxRx3 CRISPR knockouts · CORUM recall | .607 -> .626 | Helped | Masked Autoencoders for Microscopy are Scalable Learners of Cellular Biology (CVPR 2024 · 46 citations) |
| Add MMAE pre-training | OLAT test set · MAE | 0.1023 -> 0.0933 | Helped | SwitchLight: Co-design of Physics-driven Architecture and Pre-training Framework for Human Portrait Relighting (CVPR 2024 · 35 citations) |
| Add un-shared network parameters across views (condition 1) to the MVC objective | BDGP and NoisyBDGP · ACC | BDGP 66.0 -> 98.4 when combined with Av; NoisyBDGP 60.2 -> 98.0 | Helped | Investigating and Mitigating the Side Effects of Noisy Views for Self-Supervised Clustering Algorithms in Practical Multi-View Scenarios (CVPR 2024 · 29 citations) |
| Fine-tune with CosFace instead of no fine-tuning | LFW · accuracy | 75.20 -> 88.58 | Helped | LAFS: Landmark-based Facial Self-supervised Learning for Face Recognition (CVPR 2024 · 22 citations) |
| distill using multiple teacher outputs rather than the mean teacher output | SIDD Validation · PSNR | 36.79 -> 36.98 | Helped | Exploring Efficient Asymmetric Blind-Spots for Self-Supervised Denoising in Real-World Scenarios (CVPR 2024 · 15 citations) |
| add decoupled training without spatio-temporal context | GOT10K · AO | 41.8 -> 68.6 | Helped | Decoupled Spatio-Temporal Consistency Learning for Self-Supervised Tracking (AAAI 2025 · 14 citations) |
| Add composability learning to localizability | Two downstream tasks (Fig. 8) · downstream performance | Helped | Representing Part-Whole Hierarchies in Foundation Models by Learning Localizability Composability and Decomposability from Anatomy via Self Supervision (CVPR 2024 · 13 citations) | |
| Add MSTR token recycling to ARNet Basic | Clothes-V1 · Acc.@1 | 94.12 -> 95.27 | Helped | ARNet: Self-Supervised FG-SBIR with Unified Sample Feature Alignment and Multi-Scale Token Recycling (AAAI 2025 · 8 citations) |
| increase encoder-adapter bottleneck ratio from 0.0625 to 0.25 | KITTI · AbsRel | 0.092 -> 0.090 | Helped | PPEA-Depth: Progressive Parameter-Efficient Adaptation for Self-Supervised Monocular Depth Estimation (AAAI 2024 · 8 citations) |
| Add GR2R recorruption with third-order moment matching instead of standard R2R second-moment matching | Additive log-Rayleigh noise · PSNR | 25.32 ± 0.79 -> 29.93 ± 1.50 | Helped | Generalized Recorrupted-to-Recorrupted: Self-Supervised Learning Beyond Gaussian Noise (CVPR 2025 · 7 citations) |
What reviewers will expect
Datasets and baselines named in the 24 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- ImageNet · 3 of 24 papers
- COCO · 2 of 24 papers
- ETH3D · 2 of 24 papers
- ImageNet-1K · 2 of 24 papers
- Caltech101 · 2 of 24 papers
- KITTI · 2 of 24 papers
- DrivingStereo · 2 of 24 papers
- ImageNet 100 · 2 of 24 papers
Baselines
- Monodepth2 · 3 of 24 papers
- MonoViT · 2 of 24 papers
- HR-Depth · 2 of 24 papers
- SimCLR · 2 of 24 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 82 papers; from OpenAlex and the papers' affiliations).
- Nanyang Technological University · 4 papers
- Hong Kong University of Science and Technology · 4 papers
- Shanghai AI Laboratory · 3 papers
- Carnegie Mellon University · 3 papers
- New York University · 2 papers
- Sun Yat-sen University · 2 papers
- Tsinghua University · 2 papers
- A*STAR · 2 papers
- Shanghai Jiao Tong University · 2 papers
- Peking University · 2 papers
Use it
Download BibTeX of 40 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Self-supervision in supervised pipelines (121 papers)
- Self-supervised neuroimaging representations (98 papers)
- Self-supervised speech and brain decoding (56 papers)
Elsewhere in the topic
- Invariance-based self-supervised learning (168 papers)
- Data-efficient and robust self-supervision (122 papers)
- Contrastive versus non-contrastive theory (74 papers)
- Inductive bias and minimal representations (66 papers)
- Masked autoencoder pretraining (48 papers)
- Empirical contrastive visual learning (38 papers)
- Representation evaluation and trade-offs (37 papers)
- Self-supervised speech representations (24 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).