AI Paper Atlas

Research map / Label-efficient and robust learning / Self-supervised and contrastive

The hidden uniform cluster prior in self-supervised learning

Mido Assran, Randall Balestriero, Quentin Duval, Florian Bordes, Ishan Misra, Piotr Bojanowski, Pascal Vincent, Michael Rabbat, Nicolas Ballas

ICLR 2023 · Label-efficient and robust learning › Self-supervised and contrastive · approach: Invariance-based self-supervised learning · 13 citations (OpenAlex)

Read the paper

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
class-imbalanced mini-batch sampling instead of class-balanced sampling during pretraining (ablation)SimCLR, MSN, VICReg (volume-maximization SSL) · ImageNet-1K and downstream transferdownstream accuracy drop: -11.2 (SimCLR), -17.7 (MSN), -17.7 (VICReg) on ImageNetHurt
class-imbalanced mini-batch sampling instead of class-balanced sampling (ablation)MAE, data2vec · ImageNet-1K and downstream transferdownstream accuracy drop: -0.1 (MAE), -0.8 (data2vec) on ImageNetNo clear effect
replace uniform feature prior with the true iNat18 class prior (ablation)PMSN on iNat18 · iNat18 pretraining, downstream transferdownstream accuracy/score delta: +3.0 iNat18, +0.9 CIFAR100, +1.4 Places205, +2.1 Clevr/Count, +2.3 Clevr/Dist, +2.7 KITTIImproved
power-law prior (tau=0.25) instead of uniform prior (ablation)PMSN on iNat18 · iNat18 pretraining, downstream transferdownstream accuracy delta: +1.0 iNat18, +0.7 CIFAR100, +0.8 Places205, +1.7 Clevr/Count, +2.1 Clevr/Dist, +0.0 KITTIImproved
power-law prior (tau=0.25) instead of uniform prior (ablation)PMSN on ImageNet-1K · ImageNet pretraining, downstream transferdownstream accuracy delta: -19.9 iNat18, -17.8 CIFAR100, -15.7 Places205, -2.9 Clevr/Count, -6.7 KITTI; +8.0 Clevr/DistHurt
power-law prior instead of uniform prior (ablation)PMSN · Clevr/Dist and KITTI transferdepth prediction delta: +8.0 Clevr/Dist on ImageNet but -6.7 KITTI on ImageNetMixed

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy