Research map / Label-efficient and robust learning / Self-supervised and contrastive
Understanding Dimensional Collapse in Contrastive Self-supervised Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace SimCLR's 1-layer linear projector with DirectCLR's fixed-subvector loss | SimCLR with ResNet-50 · ImageNet | linear probe top-1 accuracy: 61.1 -> 62.7 | Helped |
| Replace the trainable projector with a 2-layer nonlinear projector | SimCLR with ResNet-50 · ImageNet | linear probe top-1 accuracy: 61.1 -> 66.5 | Helped |
| Remove the projector (ablation) | SimCLR with ResNet-50 · ImageNet | linear probe top-1 accuracy: 61.1 -> 51.5 | Hurt |
| Constrain the projector to be orthogonal rather than removing it (ablation) | SimCLR with ResNet-50 · ImageNet | top-1 accuracy: 51.5 -> 52.2 | No clear effect |
| Constrain the trainable projector to be diagonal (ablation) | SimCLR with ResNet-50 · ImageNet | top-1 accuracy: 61.1 -> 60.2 | No clear effect |
| Use a fixed low-rank projector instead of a trainable linear projector (ablation) | SimCLR with ResNet-50 · ImageNet | top-1 accuracy: 61.1 -> 62.3 | Helped |
| Use a fixed low-rank diagonal projector instead of a trainable linear projector (ablation) | SimCLR with ResNet-50 · ImageNet | top-1 accuracy: 61.1 -> 62.7 | Helped |
| Replace the fixed low-rank projector with a fixed low-rank diagonal projector (ablation) | SimCLR with ResNet-50 · ImageNet | top-1 accuracy: 62.3 -> 62.7 | No clear effect |
| Evaluate only the directly optimized sub-vector instead of the entire DirectCLR representation (ablation) | DirectCLR · ImageNet | linear probe accuracy: 62.7 -> 47.9% | Hurt |
| Randomly select loss features each iteration instead of using a fixed sub-vector (ablation) | SimCLR without projector · ImageNet | accuracy: 62.7 -> 43.0% | Hurt |
Closest papers by meaning
- A Simple Framework for Contrastive Learning of Visual Representations (ICML 2020 · similarity 0.90)
- Collapse-Proof Non-Contrastive Self-Supervised Learning (ICML 2025 · similarity 0.89)
- Understanding self-supervised learning dynamics without contrastive pairs (ICML 2021 · similarity 0.88)
- Self-Supervised Contrastive Learning is Approximately Supervised Contrastive Learning (NeurIPS 2025 · similarity 0.87)
- Preventing Dimensional Collapse in Self-Supervised Learning via Orthogonality Regularization (NeurIPS 2024 · similarity 0.87)
- Engineering the Neural Collapse Geometry of Supervised-Contrastive Loss (Student Abstract) (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.