Research map / Label-efficient and robust learning / Self-supervised and contrastive
How Useful Is Self-Supervised Pretraining for Visual Tasks?
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use self-supervised pretraining before finetuning rather than train from scratch | Self-supervised methods across downstream settings · Synthetic visual tasks, low-label regimes | accuracy | Helped |
| Increase the number of labeled examples for finetuning | Self-supervised pretrained models · Across downstream settings | label utility | Hurt |
| Choose pretraining method for the downstream task rather than relying on one overall ranking | Self-supervised pretraining · Object classification, pose estimation, segmentation, and depth estimation | accuracy | Mixed |
| Introduce random colors or textures (ablation) | CMC pretraining · Object classification across paired dataset variations | label utility | Helped |
| Introduce viewpoint changes (ablation) | CMC pretraining · Object classification across paired dataset variations | label utility | Hurt |
| Introduce random textures (ablation) | AMDIM pretraining · Object classification across paired dataset variations | label utility | Hurt |
| Introduce viewpoint changes (ablation) | AMDIM pretraining · Object classification across paired dataset variations | label utility | Helped |
| Add image-variation factors (ablation) | VAE pretraining · Object classification across dataset variations | label utility | Hurt |
| Use a larger ResNet50 backbone instead of ResNet9 (ablation) | Self-supervised pretraining · Object classification | label utility | Helped |
| Use CMC with the larger backbone (ablation) | CMC · Object classification, low-label regime | accuracy | Helped |
| Freeze more network blocks during finetuning (ablation) | Self-supervised pretrained ResNet9 · Downstream tasks | accuracy | Hurt |
| Freeze some network blocks instead of fully finetuning (ablation) | Self-supervised pretrained ResNet9 · Low-label regimes (100-250 samples) | accuracy | Mixed |
Closest papers by meaning
- Probing the Mid-level Vision Capabilities of Self-Supervised Learning (CVPR 2025 · similarity 0.87)
- Beyond Random Augmentations: Pretraining with Hard Views (ICLR 2025 · similarity 0.87)
- Is ImageNet worth 1 video? Learning strong image encoders from 1 long unlabelled video (ICLR 2024 · similarity 0.87)
- Learning What to Predict: Downstream-Guided Task Design for Continued Pretraining (NeurIPS 2026 · similarity 0.87)
- Harnessing small projectors and multiple views for efficient vision pretraining (NeurIPS 2024 · similarity 0.87)
- Dataset Distillation for Pre-Trained Self-Supervised Vision Models (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.