Research map / Label-efficient and robust learning / Self-supervised and contrastive
Exploring the Equivalence of Siamese Self-Supervised Learning via a Unified Gradient Framework
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a momentum target instead of a stop-gradient target (ablation) | contrastive, asymmetric, and feature-decorrelation gradient forms · ImageNet linear evaluation, 100-epoch pre-training | accuracy: 67.6 -> 70.0; 67.9 -> 70.2; 67.6 -> 69.8 | Helped |
| Use a momentum target only for the positive gradient (ablation) | contrastive, asymmetric, and feature-decorrelation gradient forms · ImageNet linear evaluation, 100-epoch pre-training | accuracy: 67.6 -> 70.1; 67.9 -> 70.3; 67.6 -> 69.8 | Helped |
| Replace the predictor matrix with identity and dynamic balance factor with a constant (ablation) | BYOL gradient form · ImageNet linear evaluation, 100-epoch pre-training | accuracy: 70.2 -> 70.3 | No clear effect |
| Replace Barlow Twins' positive-gradient matrix A with identity (ablation) | Barlow Twins gradient form · ImageNet linear evaluation, 100-epoch pre-training | accuracy: 69.0 -> 69.7 | Helped |
| Use L2 normalization instead of batch normalization (ablation) | Barlow Twins gradient form · ImageNet linear evaluation, 100-epoch pre-training | accuracy: 69.7 -> 70.0 | Helped |
| Use L2 normalization and remove the additional negative-gradient term (ablation) | VICReg gradient form · ImageNet linear evaluation, 100-epoch pre-training | accuracy: 70.0 -> 69.8 | No clear effect |
| Add CutMix (ablation) | UniGrad · ImageNet linear evaluation, 100-epoch pre-training | accuracy: 70.3 -> 71.2 | Helped |
| Add multi-crop (ablation) | UniGrad · ImageNet linear evaluation, 100-epoch pre-training | accuracy: 70.3 -> 71.7 | Helped |
| Add CutMix and multi-crop (ablation) | UniGrad · ImageNet linear evaluation, 100-epoch pre-training | accuracy: 70.3 -> 72.3 | Helped |
| Increase projector depth from 1 to 3 (ablation) | UniGrad · ImageNet linear evaluation, 100-epoch pre-training | accuracy: 60.7 -> 70.3 | Helped |
| Increase projector depth from 3 to 5 (ablation) | UniGrad · ImageNet linear evaluation, 100-epoch pre-training | accuracy: 70.3 -> 69.8 | Hurt |
| Increase projector width from 1024 to 16384 (ablation) | UniGrad · ImageNet linear evaluation, 100-epoch pre-training | accuracy: 68.3 -> 71.2 | Helped |
Closest papers by meaning
- Bridging the Gap from Asymmetry Tricks to Decorrelation Principles in Non-contrastive Self-supervised Learning (NeurIPS 2022 · similarity 0.91)
- A Simple Framework for Contrastive Learning of Visual Representations (ICML 2020 · similarity 0.90)
- Exploring Simple Siamese Representation Learning (CVPR 2021 · similarity 0.90)
- Revisiting Self-Supervised Visual Representation Learning (CVPR 2019 · similarity 0.90)
- Collapse-Proof Non-Contrastive Self-Supervised Learning (ICML 2025 · similarity 0.88)
- A Theoretical Analysis of Self-Supervised Learning for Vision Transformers (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.