Research map / Label-efficient and robust learning / Self-supervised and contrastive
On the duality between contrastive and non-contrastive self-supervised learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Tune SimCLR using VICReg's projector, temperature 0.15, and base learning rate 0.5 | SimCLR · ImageNet, 100-epoch linear evaluation | top-1 accuracy: 66.33% -> 68.6% | Helped |
| Use the VICReg projector and tuned hyperparameters rather than original SimCLR hyperparameters | SimCLR · ImageNet, 100-epoch linear evaluation | top-1 accuracy: 66.33% -> 68.6% | Helped |
| Replace VICReg covariance regularization with LogSumExp exponential covariance regularization (ablation) | VICReg · ImageNet, 100-epoch online linear evaluation, projector 8192-8192-d | top-1 accuracy: 68.13% -> 68.00% | No clear effect |
| Transpose embeddings for variance and covariance regularization (ablation) | VICReg-exp · ImageNet, 100-epoch evaluation | top-1 accuracy | No clear effect |
| Use a larger projector, 8192-8192-d instead of 2048-d (ablation) | VICReg and VICReg-ctr · ImageNet, 100-epoch evaluation | top-1 accuracy: +3.5-4 points | Helped |
| Increase projector parameter count (ablation) | studied methods · ImageNet, 100-epoch evaluation | top-1 accuracy | Helped |
| Tune VICReg for 256-dimensional embeddings (ablation) | VICReg · ImageNet, 100-epoch linear evaluation, 256-dimensional embeddings | top-1 accuracy: 55.9% -> 61.36% | Helped |
| Use a bigger projector for 256-dimensional embeddings (ablation) | VICReg · ImageNet, 100-epoch linear evaluation, 256-dimensional embeddings | top-1 accuracy: 61.36% -> 65.01% | Helped |
| Center dimensions in SimCLR (ablation) | SimCLR · ImageNet, 100-epoch evaluation | top-1 accuracy | No clear effect |
| Standardize dimensions instead of normalizing embeddings (ablation) | SimCLR · ImageNet, 100-epoch evaluation | top-1 accuracy | No clear effect |
| Use dimension standardization (ablation) | SimCLR · ImageNet, 1000-epoch linear evaluation | top-1 accuracy: 72.6% compared to 73.3% | Mixed |
| Use absolute rather than standard cosine similarities (ablation) | SimCLR · ImageNet, linear evaluation | top-1 accuracy: 68.61% -> 68.71% | Helped |
Closest papers by meaning
- A Simple Framework for Contrastive Learning of Visual Representations (ICML 2020 · similarity 0.92)
- When Does Contrastive Visual Representation Learning Work? (CVPR 2022 · similarity 0.92)
- Understanding self-supervised learning dynamics without contrastive pairs (ICML 2021 · similarity 0.91)
- Self-Contrastive Learning: Single-Viewed Supervised Contrastive Framework Using Sub-network (AAAI 2023 · similarity 0.91)
- Self-Supervised Contrastive Learning is Approximately Supervised Contrastive Learning (NeurIPS 2025 · similarity 0.90)
- Collapse-Proof Non-Contrastive Self-Supervised Learning (ICML 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.