AI Paper Atlas

Research map / Label-efficient and robust learning / Self-supervised and contrastive

Divide and Contrast: Self-Supervised Learning From Uncurated Data

Yonglong Tian, Olivier J. Hénaff, Aäron van den Oord

ICCV 2021 · Label-efficient and robust learning › Self-supervised and contrastive · approach: Contrastive versus non-contrastive theory · 70 citations (OpenAlex)

This paper tried a common change: Train longer

train longer (1000 to 3000 epochs) with MoCLR · ImageNet linear (pretrained on ImageNet) · top-1 accuracy (%): 74.3 -> 74.5 No clear effect

What 387 accepted papers report for this change

See this change in detailMatched by the wording of each paper's own result, read by AI from the full text. Papers report gains more often than failures, so this shows what gets published, not the chance a change works.

Read the paper

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add Divide and Contrast (cluster, local experts, distillation) to MoCLRMoCLR, ResNet-50 · ImageNet linear (pretrained on JFT-300M, 3000 epochs)top-1 accuracy (%): 67.4 -> 69.8Improved
add Divide and Contrast to MoCLRMoCLR, ResNet-50 · ImageNet linear (pretrained on YFCC100M, 3000 epochs)top-1 accuracy (%): 65.7 -> 67.8Improved
add Divide and Contrast to MoCLRMoCLR, ResNet-200x2 · ImageNet linear (pretrained on JFT-300M, 3000 epochs)top-1 accuracy (%): 74.2 -> 77.3Improved
add Divide and Contrast to MoCLRMoCLR, ResNet-50 · ImageNet linear (pretrained on ImageNet, 3000 epochs)top-1 accuracy (%): 74.5 -> 75.8Improved
train longer (1000 to 3000 epochs) with MoCLRMoCLR, ResNet-50 · ImageNet linear (pretrained on ImageNet)top-1 accuracy (%): 74.3 -> 74.5No clear effect
train longer (1000 to 3000 epochs) with BYOLBYOL, ResNet-50 · ImageNet linear (pretrained on ImageNet)top-1 accuracy (%): 74.3 -> 73.9Hurt
add DnC to MoCLR for semi-supervised learningMoCLR, ResNet-50 · ImageNet semi-supervised (1% labels)top-1 accuracy (%): 53.0 -> 59.9Improved
add DnC to MoCLR for fine-grained linear transferMoCLR-5k, ResNet-50 · 12-dataset linear transfer, average (pretrained on JFT-300M)average accuracy (%): 71.7 -> 76.3Improved
add DnC to MoCLR for depth estimation (YFCC100M)MoCLR-5k · NYU v2 depth estimationdelta < 1.25 accuracy (%): 86.7 -> 86.2No clear effect
train experts on the full dataset instead of local clusters (ablation)DnC · ImageNet linear (DnC, 3000 epochs)top-1 accuracy (%): 74.3 -> 75.8Improved
partition by clustering instead of random subsets (ablation)DnC · ImageNet linear (DnC, 3000 epochs)top-1 accuracy (%): 73.1 -> 75.8Improved
distill from both base model and local experts instead of base only (ablation)DnC · ImageNet linear (DnC, 3000 epochs)top-1 accuracy (%): 74.5 -> 75.8Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy