AI Paper Atlas

Research map / Label-efficient and robust learning / Self-supervised and contrastive

Contrasting Contrastive Self-Supervised Representation Learning Pipelines

Klemen Kotar, Gabriel Ilharco, Ludwig Schmidt, Kiana Ehsani, Roozbeh Mottaghi

ICCV 2021 · Label-efficient and robust learning › Self-supervised and contrastive · approach: Contrastive versus non-contrastive theory · 32 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use a self-supervised encoder instead of a supervised ImageNet encoderfrozen ResNet50 encoder · 20 end tasks (all except ImageNet v1/v2 and Pets classification)end-task performanceImproved
pre-train on ImageNet-1/4-Log (exponentially unbalanced) instead of balanced ImageNet-1/4 (ablation)SwAV and MoCov2 encoders · 20 end tasksaverage end task score: p-value = 0.0101, average end task score improvement of 1.53%Improved
pre-train on ImageNet-1/2-Lin (linearly unbalanced) instead of balanced ImageNet-1/2 (ablation)SwAV and MoCov2 encoders · 20 end tasksend task score: p-value = 0.0777No clear effect
pre-train on a dataset similar to the end task data (ablation)MoCov2 and SwAV encoders · SUN397, Caltech101, Kinetics action prediction, Taskonomy depthend task performanceImproved
pre-train on the mixed Combination dataset instead of a dataset matched to the end task (ablation)MoCov2 and SwAV encoders · matched end tasksend task performanceHurt
use SwAV instead of MoCov2ResNet50 encoders trained 200 epochs · image-level semantic tasks vs pixelwise tasksend task performanceMixed
add SwAV MultiCrop pre-processing to MoCov2 (ablation)MoCov2 on ImageNet · 20 end tasksend task performanceMixed
use SimSiam instead of SwAVResNet50 encoder · 20 end tasksnormalized end task scoreNo clear effect
use Exemplar-v2 supervised training instead of vanilla supervisedsupervised ImageNet encoder · 20 end tasksend task performanceImproved
use self-supervised encoders on structural tasks rather than semantic tasksself-supervised encoders vs supervised ImageNet · structural vs semantic end tasksnormalized performance over supervisedImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy