Research map / Label-efficient and robust learning / Self-supervised and contrastive
Understanding Collapse in Non-Contrastive Siamese Representation Learning
Alexander C. Li, Alexei A. Efros, Deepak Pathak
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| switch from multiple-pass to single-pass continual training (ablation) | SimSiam ResNet-18 · ImageNet-1k linear probe (SimSiam) | Linear probe accuracy: 30.0 -> 44.5 | Improved |
| switch from multiple-pass to single-pass continual training (ablation) | SimSiam ResNet-34 · ImageNet-1k linear probe (SimSiam) | Linear probe accuracy: 16.8 -> 45.0 | Improved |
| switch from multiple-pass to single-pass continual training (ablation) | SimSiam ResNet-50 · ImageNet-1k linear probe (SimSiam) | Linear probe accuracy: 68.1 -> 55.9 | Hurt |
| add cumulative data ordering instead of multiple pass (ablation) | SimSiam ResNet-18 · ImageNet-1k linear probe (SimSiam) | Linear probe accuracy: 30.0 -> 33.0 | Improved |
| hybrid: continual for first 40 chunks then multi-epoch (ablation) | SimSiam ResNet-18 · ImageNet-1k linear probe (SimSiam) | Linear probe accuracy: 30.0 -> 48.3 | Improved |
| hybrid: continual for first 40 chunks then multi-epoch (ablation) | SimSiam ResNet-50 · ImageNet-1k linear probe (SimSiam) | Linear probe accuracy: 68.1 -> 67.6 | No clear effect |
| increase depth from ResNet-18 to ResNet-34 with Basic blocks (ablation) | SimSiam · ImageNet-1k linear probe (SimSiam) | Linear probe accuracy: 30.0 -> 16.8 | Hurt |
| double width of Bottleneck-26 (ablation) | SimSiam · ImageNet-1k linear probe (SimSiam) | Linear probe accuracy: 61.7 -> 62.6 | Improved |
| use Bottleneck blocks (wider representation) in place of Basic blocks at depth 50 (ablation) | SimSiam · ImageNet-1k linear probe (SimSiam) | Linear probe accuracy: 17.5 -> 68.1 | Improved |
| add 10-epoch learning rate warmup (ablation) | SimSiam multiple pass ResNet-18 · ImageNet-1k linear probe (SimSiam) | Linear probe accuracy: 30.0 -> 28.4 | Hurt |
| add 10-epoch learning rate warmup (ablation) | SimSiam multiple pass ResNet-34 · ImageNet-1k linear probe (SimSiam) | Linear probe accuracy: 16.8 -> 35.8 | Improved |
| hybrid training on Places365 (ablation) | SimSiam ResNet-18 · Places365 linear probe | Linear probe accuracy: 30.0 -> 35.0 | Improved |
Closest papers by meaning
- Implicit Contrastive Representation Learning with Guided Stop-gradient (NeurIPS 2023 · similarity 0.91)
- Symmetric Neural-Collapse Representations with Supervised Contrastive Loss: The Impact of ReLU and Batching (ICLR 2024 · similarity 0.87)
- Self-Supervised Contrastive Learning is Approximately Supervised Contrastive Learning (NeurIPS 2025 · similarity 0.87)
- Collapse-Proof Non-Contrastive Self-Supervised Learning (ICML 2025 · similarity 0.87)
- Engineering the Neural Collapse Geometry of Supervised-Contrastive Loss (Student Abstract) (AAAI 2024 · similarity 0.87)
- PEIRA: Learning Predictive Encoders through Inter-View Regressor Alignment (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
