Research map / Label-efficient and robust learning / Self-supervised and contrastive
Exploring Simple Siamese Representation Learning
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add stop-gradient to the target branch (ablation) | SimSiam · ImageNet linear evaluation | accuracy (%): 0.1 -> 67.7 | Helped |
| Add a trainable prediction MLP (ablation) | SimSiam · ImageNet linear evaluation | accuracy (%): 0.1 -> 67.7 | Helped |
| Train the predictor instead of fixing it at random initialization (ablation) | SimSiam · ImageNet linear evaluation | accuracy (%): 1.5 -> 67.7 | Helped |
| Do not decay the predictor learning rate (ablation) | SimSiam · ImageNet linear evaluation | accuracy (%): 67.7 -> 68.1 | No clear effect |
| Reduce batch size from 256 to 64 (ablation) | SimSiam · ImageNet linear evaluation | accuracy (%): 68.1 -> 66.1 | Hurt |
| Increase batch size from 256 to 4096 (ablation) | SimSiam · ImageNet linear evaluation | accuracy (%): 68.1 -> 64.0 | Hurt |
| Add batch normalization to hidden MLP layers (ablation) | SimSiam · ImageNet linear evaluation | accuracy (%): 34.6 -> 67.4 | Helped |
| Add batch normalization to the projection MLP output (ablation) | SimSiam · ImageNet linear evaluation | accuracy (%): 67.4 -> 68.1 | Helped |
| Add batch normalization to the prediction MLP output (ablation) | SimSiam · ImageNet linear evaluation | training stability | Hurt |
| Replace cosine similarity with cross-entropy similarity (ablation) | SimSiam · ImageNet linear evaluation | accuracy (%): 68.1 -> 63.2 | Hurt |
| Remove loss symmetrization (ablation) | SimSiam · ImageNet linear evaluation | accuracy (%): 68.1 -> 64.8 | Hurt |
| Increase output dimension from 256 to 2048 (ablation) | SimSiam · ImageNet linear evaluation | accuracy (%): 65.3 -> 68.1 | Helped |
Closest papers by meaning
- On the Importance of Asymmetry for Siamese Representation Learning (CVPR 2022 · similarity 0.90)
- Exploring the Equivalence of Siamese Self-Supervised Learning via a Unified Gradient Framework (CVPR 2022 · similarity 0.90)
- Implicit Contrastive Representation Learning with Guided Stop-gradient (NeurIPS 2023 · similarity 0.89)
- Siamese Image Modeling for Self-Supervised Vision Representation Learning (CVPR 2023 · similarity 0.89)
- Crafting Better Contrastive Views for Siamese Representation Learning (CVPR 2022 · similarity 0.87)
- Tracing Representation Progression: Analyzing and Enhancing Layer-Wise Similarity (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.