AI Paper Atlas

Research map / Label-efficient and robust learning / Self-supervised and contrastive

Understanding Collapse in Non-Contrastive Siamese Representation Learning

Alexander C. Li, Alexei A. Efros, Deepak Pathak

ECCV 2022 · Label-efficient and robust learning › Self-supervised and contrastive · approach: Contrastive versus non-contrastive theory · 34 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
switch from multiple-pass to single-pass continual training (ablation)SimSiam ResNet-18 · ImageNet-1k linear probe (SimSiam)Linear probe accuracy: 30.0 -> 44.5Improved
switch from multiple-pass to single-pass continual training (ablation)SimSiam ResNet-34 · ImageNet-1k linear probe (SimSiam)Linear probe accuracy: 16.8 -> 45.0Improved
switch from multiple-pass to single-pass continual training (ablation)SimSiam ResNet-50 · ImageNet-1k linear probe (SimSiam)Linear probe accuracy: 68.1 -> 55.9Hurt
add cumulative data ordering instead of multiple pass (ablation)SimSiam ResNet-18 · ImageNet-1k linear probe (SimSiam)Linear probe accuracy: 30.0 -> 33.0Improved
hybrid: continual for first 40 chunks then multi-epoch (ablation)SimSiam ResNet-18 · ImageNet-1k linear probe (SimSiam)Linear probe accuracy: 30.0 -> 48.3Improved
hybrid: continual for first 40 chunks then multi-epoch (ablation)SimSiam ResNet-50 · ImageNet-1k linear probe (SimSiam)Linear probe accuracy: 68.1 -> 67.6No clear effect
increase depth from ResNet-18 to ResNet-34 with Basic blocks (ablation)SimSiam · ImageNet-1k linear probe (SimSiam)Linear probe accuracy: 30.0 -> 16.8Hurt
double width of Bottleneck-26 (ablation)SimSiam · ImageNet-1k linear probe (SimSiam)Linear probe accuracy: 61.7 -> 62.6Improved
use Bottleneck blocks (wider representation) in place of Basic blocks at depth 50 (ablation)SimSiam · ImageNet-1k linear probe (SimSiam)Linear probe accuracy: 17.5 -> 68.1Improved
add 10-epoch learning rate warmup (ablation)SimSiam multiple pass ResNet-18 · ImageNet-1k linear probe (SimSiam)Linear probe accuracy: 30.0 -> 28.4Hurt
add 10-epoch learning rate warmup (ablation)SimSiam multiple pass ResNet-34 · ImageNet-1k linear probe (SimSiam)Linear probe accuracy: 16.8 -> 35.8Improved
hybrid training on Places365 (ablation)SimSiam ResNet-18 · Places365 linear probeLinear probe accuracy: 30.0 -> 35.0Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy