Research map / Label-efficient and robust learning / Self-supervised and contrastive
Contrastive versus non-contrastive theory
Analyzes dynamics and duality of contrastive and non-contrastive self-supervised learning.
74 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2020–2026) · an approach in Self-supervised and contrastive, keyword cluster “supervised contrastive · contrastive self · negative”
State History Benchmarks Tried Checklist Preprints Who Export
Where it stands
Quiet None of the tracked benchmarks has a report from 2025–26.
29 papers in 2023–24 and 15 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 9.0% to 4.7%.
No longer reported by papers here: ImageNet-100 (top-1 accuracy, last in 2024).
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2020 · Supervised Contrastive Learning (NeurIPS 2020 · 113 citations)
- 2021 · Dual-Stream Multiple Instance Learning Network for Whole Slide Image Classification With Self-Supervised Contrastive Learning (CVPR 2021 · 888 citations)
- 2022 · Contrastive Learning from Extremely Augmented Skeleton Sequences for Self-Supervised Action Recognition (AAAI 2022 · 180 citations)
- 2023 · Change-Aware Sampling and Contrastive Learning for Satellite Images (CVPR 2023 · 94 citations)
- 2024 · SCD-Net: Spatiotemporal Clues Disentanglement Network for Self-Supervised Skeleton-Based Action Recognition (AAAI 2024 · 24 citations)
- 2025 · Self-supervised Trusted Contrastive Multi-view Clustering with Uncertainty Refined (AAAI 2025 · 9 citations)
- 2026 · Towards Better Code Understanding in Decoder-Only Models with Contrastive Learning (AAAI 2026 · 1 citation)
Earliest papers here
- Supervised Contrastive Learning (NeurIPS 2020 · 113 citations)
- Dual-Stream Multiple Instance Learning Network for Whole Slide Image Classification With Self-Supervised Contrastive Learning (CVPR 2021 · 888 citations)
- Shot Contrastive Self-Supervised Learning for Scene Boundary Detection (CVPR 2021 · 51 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
ImageNet-100 · top-1 accuracy (higher is better)
4 papers, 2021–2024 · not reported since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2021 | 79.02 | Whitening for Self-Supervised Representation Learning (ICML 2021 · 42 citations) |
| 2022 | 82.6 | Learning Weakly-supervised Contrastive Representations (ICLR 2022 · 2 citations) |
| 2024 | 87.43 | On the Effectiveness of Supervision in Asymmetric Non-Contrastive Learning (ICML 2024 · 0 citations) |
What has been tried
60 papers here report 689 measured changes (a component added, removed or swapped, compared with a baseline): helped 64%, no clear effect 9%, hurt 24%, mixed 3%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| remove the feature extractor | NTU-60 x-sub · Top-1 accuracy | 86.6 -> 80.0 | Hurt | SCD-Net: Spatiotemporal Clues Disentanglement Network for Self-Supervised Skeleton-Based Action Recognition (AAAI 2024 · 24 citations) |
| Add uncertainty-refined contrastive learning (LCL) and view fusion consistency loss (LCE) to LC | Caltech-2V · ACC | 48.9 -> 71.9 | Helped | Self-supervised Trusted Contrastive Multi-view Clustering with Uncertainty Refined (AAAI 2025 · 9 citations) |
| Add dropout in the linear embedding without prediction alignment | ImageNet-1K val after COCO pre-training · linear probing top-1 accuracy | 46.8% -> 46.5% | Hurt | Self-Supervised Representation Learning from Arbitrary Scenarios (CVPR 2024 · 7 citations) |
| Remove the uniformity non-collapse term while retaining invariance and equivariance losses | CIFAR10 · representation collapse | Hurt | Structuring Representation Geometry with Rotationally Equivariant Contrastive Learning (ICLR 2024 · 1 citation) | |
| Increase dictionary size from c=10 or c=f to c=16384 | SVHN, CIFAR-10, CIFAR-100 · NMI and accuracy | SVHN NMI: 0.11±0.01 -> 0.29±\textbf{0.00}; CIFAR-10 NMI: 0.28±0.02 -> \textbf{0.35}±\textbf{0.00}; CIFAR | Helped | Collapse-Proof Non-Contrastive Self-Supervised Learning (ICML 2025 · 0 citations) |
| Increase the number of augmentation views from 2 to 4 | ImageNet-1K 30% linear evaluation · top 1 accuracy | 55.17 -> 58.29 | Helped | Contrastive Self-Supervised Learning As Neural Manifold Packing (NeurIPS 2025 · 0 citations) |
| Learn per-example ζ instead of fixing ζ to ζ₀ | Downstream retrieval and classification · performance | Helped | On Discriminative Probabilistic Modeling for Self-Supervised Representation Learning (ICLR 2025 · 0 citations) | |
| replace METRA's representation objective with a contrastive loss | Ant exploration · state coverage | No clear effect | Can a MISL Fly? Analysis and Ingredients for Mutual Information Skill Learning (ICLR 2025 · 0 citations) | |
| Set α to 0.5 rather than α=1.0 | Toy dataset · Linear probing accuracy | 45.91 -> 61.89 | Helped | On the Effectiveness of Supervision in Asymmetric Non-Contrastive Learning (ICML 2024 · 0 citations) |
| use one shared threshold instead of a separate learned threshold for each anchor | ImageNet100 semi-supervised linear evaluation · performance | Hurt | Discovering Global False Negatives On the Fly for Self-supervised Contrastive Learning (ICML 2025) | |
| Add contrastive regularization to the autoencoder | MPII · gaze error | 6.60 -> 5.76 | Helped | Bootstrap AutoEncoders With Contrastive Paradigm for Self-supervised Gaze Estimation (ICML 2024) |
What reviewers will expect
Datasets and baselines named in the 7 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- CIFAR-10 · 4 of 7 papers
- ImageNet-100 · 3 of 7 papers
- CIFAR-100 · 3 of 7 papers
- CC3M · 2 of 7 papers
- SVHN · 2 of 7 papers
- ImageNet-1K · 2 of 7 papers
Baselines
- SogCLR · 2 of 7 papers
- SimCLR · 2 of 7 papers
- Barlow Twins · 2 of 7 papers
- SwAV · 2 of 7 papers
- MoCo-V2 · 2 of 7 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 44 papers; from OpenAlex and the papers' affiliations).
- Peking University · 4 papers
- Meta · 4 papers
- Shanghai Jiao Tong University · 4 papers
- Stanford University · 3 papers
- Huawei · 3 papers
- New York University · 3 papers
- Texas A&M University - College Station · 3 papers
- Johns Hopkins University · 2 papers
- Amazon · 2 papers
- Google · 2 papers
Use it
Download BibTeX of 40 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Inductive bias and minimal representations (66 papers)
- Empirical contrastive visual learning (38 papers)
- Representation evaluation and trade-offs (37 papers)
Elsewhere in the topic
- Invariance-based self-supervised learning (168 papers)
- Pretext-task visual pre-training (136 papers)
- Data-efficient and robust self-supervision (122 papers)
- Self-supervision in supervised pipelines (121 papers)
- Self-supervised neuroimaging representations (98 papers)
- Self-supervised speech and brain decoding (56 papers)
- Masked autoencoder pretraining (48 papers)
- Self-supervised speech representations (24 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).