Research map / Label-efficient and robust learning / Self-supervised and contrastive
The Trade-off between Universality and Label Efficiency of Representations from Contrastive Learning
Zhenmei Shi, Jiefeng Chen, Kunyang Li, Jayaram Raghuram, Xi Wu, Yingyu Liang, Somesh Jha
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add contrastive regularization during fine-tuning | MoCo v2 on CIFAR-10 · CIFAR-10, pre-training data expanded from CINIC-10 to CINIC-10 + SVHN + GTSRB + ImageNet32 | test accuracy: 92.92± 0.06 -> 93.66± 0.12 | Improved |
| Add contrastive regularization during fine-tuning | CLIP ViT-L · ImageNet | test accuracy: 83.65± 0.01 -> 84.94± 0.09 | Improved |
| Add contrastive regularization during fine-tuning | MoCo v3 ViT-B · GTSRB | test accuracy: 76.45± 0.29 -> 81.28± 0.10 | Improved |
| Add contrastive regularization during fine-tuning | CLIP ViT-L · SVHN | test accuracy: 78.22± 0.18 -> 78.72± 0.37 | Improved |
| Add contrastive regularization during fine-tuning | CLIP ViT-L · GTSRB | test accuracy: 90.74± 0.06 -> 92.01± 0.28 | Improved |
| Add contrastive regularization during fine-tuning | MoCo v3 ViT-B · CIFAR-10 | test accuracy: 96.17± 0.12 -> 96.71± 0.10 | Improved |
| Add contrastive regularization during fine-tuning | MoCo v3 ViT-B · SVHN | test accuracy: 65.36± 0.33 -> 66.29± 0.20 | Improved |
| Add contrastive regularization during fine-tuning | SimCSE BERT · IMDB | test accuracy: 92.31± 0.26 -> 92.85± 0.03 | Improved |
| Add contrastive regularization during fine-tuning | SimCSE BERT · AGNews | test accuracy: 93.57± 0.23 -> 93.94± 0.02 | Improved |
| Increase pre-training task diversity | MoCo v2 representations · Four pre-training datasets; downstream tasks are the same datasets | average test accuracy: 18.3% to 20.1% | Improved |
| Increase pre-training task diversity | MoCo v2 representations · CIFAR-10 target task | test accuracy: 88.5% to 76.4% | Hurt |
Closest papers by meaning
- The Power of Contrast for Feature Learning: A Theoretical Analysis (JMLR 2023 · similarity 0.89)
- Mitigating Spurious Features in Contrastive Learning with Spectral Regularization (NeurIPS 2025 · similarity 0.88)
- Debiased Contrastive Learning (NeurIPS 2020 · similarity 0.88)
- A Statistical Theory of Contrastive Learning via Approximate Sufficient Statistics (NeurIPS 2025 · similarity 0.87)
- How does Labeling Error Impact Contrastive Learning? A Perspective from Data Dimensionality Reduction (ICML 2025 · similarity 0.87)
- Generalization Analysis for Supervised Contrastive Representation Learning under Non-IID Settings (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
