Research map / Label-efficient and robust learning / Self-supervised and contrastive
An Augmentation Overlap Theory of Contrastive Learning
Qi Zhang, Yifei Wang, Yisen Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase augmentation strength r from none to moderate on synthetic data (ablation) | InfoNCE contrastive learning · Synthetic two-class sphere data | linear accuracy: accuracy increases first and then falls to 50% | Mixed |
| use appropriate RandomResizedCrop strength instead of too weak (0.99,1) (ablation) | SimCLR · CIFAR-10 | linear accuracy: 25% -> 75% | Improved |
| make RandomResizedCrop too strong instead of appropriate (ablation) | SimCLR · CIFAR-10 | linear accuracy: 75% -> 29% | Hurt |
| use GARC instead of rotation accuracy as unsupervised representation metric | representation evaluation · CIFAR-10 | Pearson correlation with linear accuracy: 0.905 -> 0.954 | Improved |
| use InfoNCE loss as indicator of downstream performance | representation evaluation · CIFAR-10 | InfoNCE loss vs linear accuracy: InfoNCE 4.8 to 5.0 while accuracy 40% to 80% | No clear effect |
| replace conditional independence assumption with augmentation overlap assumption in the bound | contrastive learning theory · Upper bound of supervised loss | bound error as M grows: L_unsup(f) + e/sqrt(M) | Improved |
Closest papers by meaning
- Chaos is a Ladder: A New Theoretical Understanding of Contrastive Learning via Augmentation Overlap (ICLR 2022 · similarity 0.94)
- An Augmentation-Aware Theory for Self-Supervised Contrastive Learning (ICML 2025 · similarity 0.93)
- A Statistical Theory of Contrastive Learning via Approximate Sufficient Statistics (NeurIPS 2025 · similarity 0.89)
- Self-Supervised Contrastive Learning is Approximately Supervised Contrastive Learning (NeurIPS 2025 · similarity 0.88)
- Perfect Alignment May be Poisonous to Graph Contrastive Learning (ICML 2024 · similarity 0.87)
- Generalization Analysis for Supervised Contrastive Representation Learning under Non-IID Settings (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
