Research map / Label-efficient and robust learning / Self-supervised and contrastive
Revisiting Self-Supervised Visual Representation Learning
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a wider RevNet50 for Rotation pretraining (4× to 16×) (ablation) | Rotation self-supervision · ImageNet own validation split | accuracy: 47.3 -> 53.7 | Helped |
| Use a wider ResNet50 v1 for Relative Patch Location pretraining (4× to 8×) (ablation) | Relative Patch Location · ImageNet own validation split | accuracy: 46.8 -> 50.5 | Helped |
| Use a RevNet instead of a ResNet for Rotation pretraining at 16× (ablation) | Rotation self-supervision · ImageNet own validation split | accuracy: 47.6 -> 53.7 | Helped |
| Use a RevNet instead of a ResNet across pretext tasks (ablation) | Self-supervised representations · ImageNet own validation split | accuracy | Mixed |
| Increase CNN width (ablation) | Self-supervised CNN representations · ImageNet and Places205 | accuracy | Helped |
| Increase representation size while holding network width fixed (ablation) | Rotation RevNet50 · ImageNet classification | accuracy: 31% to 43% | Helped |
| Increase both model width and representation size (ablation) | Rotation RevNet50 · ImageNet classification | accuracy | Helped |
| Use the final pre-logits layer instead of earlier layers (ablation) | ResNet and RevNet representations · ImageNet classification | accuracy | Helped |
| Use skip connections rather than a VGG-style network (ablation) | Self-supervised visual representation learning · ImageNet classification | accuracy | Helped |
| Evaluate representations with a one-hidden-layer MLP rather than a linear model (ablation) | Self-supervised representations · ImageNet classification | accuracy | No clear effect |
| Train the linear evaluation model for many more epochs before the first learning-rate decay (ablation) | SGD linear evaluation · ImageNet classification | accuracy | Helped |
| Choose the architecture based on pretext-task accuracy across different architectures (ablation) | Self-supervised model selection · ImageNet classification | accuracy | Hurt |
Closest papers by meaning
- Efficient Self-supervised Vision Transformers for Representation Learning (ICLR 2022 · similarity 0.91)
- When Does Contrastive Visual Representation Learning Work? (CVPR 2022 · similarity 0.90)
- Learning Representations by Predicting Bags of Visual Words (CVPR 2020 · similarity 0.90)
- Self-Organizing Visual Prototypes for Non-Parametric Representation Learning (ICML 2025 · similarity 0.88)
- Harnessing small projectors and multiple views for efficient vision pretraining (NeurIPS 2024 · similarity 0.88)
- Learning from Memory: Non-Parametric Memory Augmented Self-Supervised Learning of Visual Features (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.