Research map / Label-efficient and robust learning / Recognition and long-tail
Contrastive representation theory
Analyzes unsupervised and contrastive representation learning, disentanglement and invariance.
495 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Recognition and long-tail, keyword cluster “representations · transfer · unsupervised”
State History Benchmarks Tried Baselines Checklist Preprints Who Export
Where it stands
Still moving The best reported result is still improving on 1 of 6 tracked benchmarks.
127 papers in 2023–24 and 157 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic grew, from 10.1% to 12.0%.
Open questions the data points to
- ImageNet (top-1 accuracy): best went from 78.21 (up to 2024) to 100 in 2025 — Functional Alignment Can Mislead: Examining Model Stitching (ICML 2025)
- Papers report FactorVAE on dSprites (FactorVAE score) anywhere from 0.74 to 0.819: comparisons against it depend on whose number you take.
What looks settled
- CIFAR-100 (Top-1 accuracy): best up to 2024 81.07, best in 2025–26 78.29 from 2 papers, at most 0.2% better, if at all.
No longer reported by papers here: CIFAR-10 (accuracy, last in 2024); CIFAR-100 (accuracy, last in 2024); ImageNet (accuracy, last in 2024).
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Cross-Stitch Networks for Multi-Task Learning (CVPR 2016 · 1,457 citations)
- 2017 · Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Prediction (CVPR 2017 · 653 citations)
- 2018 · Disentangling by Factorising (ICML 2018 · 670 citations)
- 2019 · Learning deep representations by mutual information estimation and maximization (ICLR 2019 · 1,723 citations)
- 2020 · Learning Attentive Pairwise Interaction for Fine-Grained Classification (AAAI 2020 · 420 citations)
- 2021 · Contrastive Clustering (AAAI 2021 · 648 citations)
- 2022 · Invariant Information Bottleneck for Domain Generalization (AAAI 2022 · 102 citations)
- 2023 · Dynamic Conceptional Contrastive Learning for Generalized Category Discovery (CVPR 2023 · 86 citations)
- 2024 · Online Boosting Adaptive Learning under Concept Drift for Multistream Classification (AAAI 2024 · 27 citations)
- 2025 · USDRL: Unified Skeleton-Based Dense Representation Learning with Multi-Grained Feature Decorrelation (AAAI 2025 · 9 citations)
- 2026 · On the Theoretical Limitations of Embedding-Based Retrieval (ICLR 2026 · 6 citations)
Earliest papers here
- Cross-Stitch Networks for Multi-Task Learning (CVPR 2016 · 1,457 citations)
- InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets (NeurIPS 2016 · 1,227 citations)
- DeepMath - Deep Sequence Models for Premise Selection (NeurIPS 2016 · 112 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
CIFAR-10 · accuracy (higher is better)
9 papers, 2020–2024 · not reported since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2020 | 86.49 | Hierarchically Robust Representation Learning (CVPR 2020 · 7 citations) |
| 2021 | 97.8 | Scalable Transfer Learning with Expert Models (ICLR 2021 · 26 citations) |
| 2023 | 99.11 | Modeling Inter-Class and Intra-Class Constraints in Novel Class Discovery (CVPR 2023 · 35 citations) |
ImageNet · top-1 accuracy (higher is better)
7 papers, 2019–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2019 | 76.17 | Local to Global Learning: Gradually Adding Classes for Training Deep Neural Networks (CVPR 2019 · 16 citations) |
| 2022 | 78.21 | MaskTune: Mitigating Spurious Correlations by Forcing to Explore (NeurIPS 2022 · 19 citations) |
| 2025 | 100 | Functional Alignment Can Mislead: Examining Model Stitching (ICML 2025) |
CIFAR-10 · top-1 accuracy (higher is better)
4 papers, 2022–2025 · one report since 2024, not better
| Year | Best so far | Set by |
|---|---|---|
| 2022 | 96.88 | Stimulative Training of Residual Networks: A Social Psychology Perspective of Loafing (NeurIPS 2022 · 3 citations) |
CIFAR-100 · accuracy (higher is better)
4 papers, 2021–2024 · not reported since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2021 | 86.6 | Neighborhood Contrastive Learning for Novel Class Discovery (CVPR 2021 · 134 citations) |
CIFAR-100 · Top-1 accuracy (higher is better)
4 papers, 2022–2025 · flat since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2022 | 81.07 | Stimulative Training of Residual Networks: A Social Psychology Perspective of Loafing (NeurIPS 2022 · 3 citations) |
ImageNet · accuracy (higher is better)
4 papers, 2021–2024 · not reported since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2021 | 90.7 | Neighborhood Contrastive Learning for Novel Class Discovery (CVPR 2021 · 134 citations) |
What has been tried
380 papers here report 3,965 measured changes (a component added, removed or swapped, compared with a baseline): helped 67%, no clear effect 9%, hurt 19%, mixed 5%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| add drift and covariate-shift adaptation | all eight benchmarks · accuracy (%) | OBAL_v1: 79.54, 72.42, 79.78, 81.42, 72.25, 62.14, 63.12, 58.02; OBAL: 90.98, 78.45, 86.78, 88.01, 79.22, 70.29, 70.71, | Helped | Online Boosting Adaptive Learning under Concept Drift for Multistream Classification (AAAI 2024 · 27 citations) |
| add attention diversity loss to the single-expert baseline | CIFAR10 known / ImageNet-R unknown · macro-F1 | 0.944 -> 0.977 | Helped | Exploring Diverse Representations for Open Set Recognition (AAAI 2024 · 25 citations) |
| Add P2I Trans | BANKING · H-score | 65.46 -> 66.70 | Helped | Transfer and Alignment Network for Generalized Category Discovery (AAAI 2024 · 12 citations) |
| Replace KL-divergence imitability metric with L2 loss | CIFAR-100 · accuracy | 93.17 -> 92.77 | Hurt | Learning from Teaching Regularization: Generalizable Correlations Should be Easy to Imitate (NeurIPS 2024 · 12 citations) |
| Add the XC matrix | NTU-60 recognition and retrieval · accuracy | +0.5% | Helped | USDRL: Unified Skeleton-Based Dense Representation Learning with Multi-Grained Feature Decorrelation (AAAI 2025 · 9 citations) |
| Add contrastive learning to the model with learnable prior and transformer fusion | Ionosphere · ma-F1 | 0.562 -> 0.603 | Helped | A Learnable Discrete-Prior Fusion Autoencoder with Contrastive Learning for Tabular Data Synthesis (AAAI 2024 · 9 citations) |
| Add the time-varying strategy | US-Airport node classification · Micro F1 | 67.25% -> 68.99% | Helped | Measuring Task Similarity and Its Implication in Fine-Tuning Graph Neural Networks (AAAI 2024 · 9 citations) |
| include LCOM (with LSEP absent and L2-norm constraint) | Digits · average accuracy | 60.89 -> 61.06 | Helped | Beyond Federated Prototype Learning: Learnable Semantic Anchors with Hyperspherical Contrast for Domain-Skewed Data (AAAI 2025 · 7 citations) |
| add infoNCE loss Lcl | BANKING · All accuracy | 77.48 -> 77.93 | Helped | A Unified Knowledge Transfer Network for Generalized Category Discovery (AAAI 2024 · 7 citations) |
| Add noisy links to information-flow-controlled pretraining | DTI DG Temporal · Pearson correlation | 0.575 -> 0.578; 0.576 -> 0.579; 0.578 -> 0.580 | Helped | Knowledge Enhanced Representation Learning for Drug Discovery (AAAI 2024 · 7 citations) |
| Use a nonlinearity with two-sided saturation instead of ReLU | Random classification tasks with P=32 inputs · target alignment | Helped | Task structure and nonlinearity jointly determine learned representational geometry (ICLR 2024 · 7 citations) | |
| Add an MLP projector for contrastive loss | SDD, unobservable length 6 · ADE/FDE | 8.23/12.94 -> 7.97/12.54 | Helped | Adapting to Observation Length of Trajectory Prediction via Contrastive Learning (CVPR 2025 · 6 citations) |
Baseline numbers that disagree
The same baseline on the same benchmark and metric, reported more than 5% apart by different papers here. Training budgets, data splits and reimplementations differ, so state which number you compare with.
| Baseline | Benchmark | Metric | Reported | Papers |
|---|---|---|---|---|
| FactorVAE | dSprites | FactorVAE score | 0.74 – 0.819 | 2 papers, e.g. ICLR 2022 and ICML 2024 |
| FactorVAE | dSprites | MIG | 0.27 – 0.295 | 2 papers, e.g. ICLR 2022 and ICML 2024 |
What reviewers will expect
Datasets and baselines named in the 95 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- CIFAR-10 · 18 of 95 papers
- CIFAR-100 · 18 of 95 papers
- ImageNet-100 · 7 of 95 papers
- MNIST · 7 of 95 papers
- ImageNet-1K · 6 of 95 papers
- FGVC-Aircraft · 5 of 95 papers
- Stanford Cars · 4 of 95 papers
- CUB-200-2011 · 4 of 95 papers
Baselines
- SimCLR · 4 of 95 papers
- K-means · 3 of 95 papers
- GCD · 3 of 95 papers
- PromptCAL · 3 of 95 papers
- SimGCD · 3 of 95 papers
- PCA · 3 of 95 papers
- Linear · 2 of 95 papers
- Cross-Entropy · 2 of 95 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 284 papers; from OpenAlex and the papers' affiliations).
- Google · 10 papers
- Massachusetts Institute of Technology · 8 papers
- Amazon · 8 papers
- Peking University · 6 papers
- Tsinghua University · 6 papers
- Zhejiang University · 6 papers
- Google DeepMind · 6 papers
- University of California, Los Angeles · 5 papers
- New York University · 5 papers
- Stanford University · 5 papers
Use it
Download BibTeX of 40 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Correspondence and causal representation learning (447 papers)
- Dense features and foundation priors (482 papers)
- Data augmentation and generalization bias (409 papers)
- Multilingual language representation pretraining (507 papers)
- Recommendation representation learning (207 papers)
Elsewhere in the topic
- Partial and multi-label learning (345 papers)
- Loss design and classifier theory (273 papers)
- Group-robust classification (251 papers)
- Learning from unlabeled and positive-unlabeled data (240 papers)
- Confidence calibration objectives (129 papers)
- Margin-based face recognition losses (124 papers)
- Supervised person re-identification (108 papers)
- Self-explaining and visual explanation networks (108 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).