AI Paper Atlas

Research map / Label-efficient and robust learning / Self-supervised and contrastive

The Trade-off between Universality and Label Efficiency of Representations from Contrastive Learning

Zhenmei Shi, Jiefeng Chen, Kunyang Li, Jayaram Raghuram, Xi Wu, Yingyu Liang, Somesh Jha

ICLR 2023 · Label-efficient and robust learning › Self-supervised and contrastive · approach: Representation evaluation and trade-offs · 1 citations (OpenAlex)

Read the paper

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Add contrastive regularization during fine-tuningMoCo v2 on CIFAR-10 · CIFAR-10, pre-training data expanded from CINIC-10 to CINIC-10 + SVHN + GTSRB + ImageNet32test accuracy: 92.92± 0.06 -> 93.66± 0.12Improved
Add contrastive regularization during fine-tuningCLIP ViT-L · ImageNettest accuracy: 83.65± 0.01 -> 84.94± 0.09Improved
Add contrastive regularization during fine-tuningMoCo v3 ViT-B · GTSRBtest accuracy: 76.45± 0.29 -> 81.28± 0.10Improved
Add contrastive regularization during fine-tuningCLIP ViT-L · SVHNtest accuracy: 78.22± 0.18 -> 78.72± 0.37Improved
Add contrastive regularization during fine-tuningCLIP ViT-L · GTSRBtest accuracy: 90.74± 0.06 -> 92.01± 0.28Improved
Add contrastive regularization during fine-tuningMoCo v3 ViT-B · CIFAR-10test accuracy: 96.17± 0.12 -> 96.71± 0.10Improved
Add contrastive regularization during fine-tuningMoCo v3 ViT-B · SVHNtest accuracy: 65.36± 0.33 -> 66.29± 0.20Improved
Add contrastive regularization during fine-tuningSimCSE BERT · IMDBtest accuracy: 92.31± 0.26 -> 92.85± 0.03Improved
Add contrastive regularization during fine-tuningSimCSE BERT · AGNewstest accuracy: 93.57± 0.23 -> 93.94± 0.02Improved
Increase pre-training task diversityMoCo v2 representations · Four pre-training datasets; downstream tasks are the same datasetsaverage test accuracy: 18.3% to 20.1%Improved
Increase pre-training task diversityMoCo v2 representations · CIFAR-10 target tasktest accuracy: 88.5% to 76.4%Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy