Research map / Label-efficient and robust learning / Self-supervised and contrastive
The Lottery Tickets Hypothesis for Supervised and Self-Supervised Pre-Training in Computer Vision Models
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Prune supervised ImageNet pre-trained weights and transfer the resulting mask | ResNet-50 ticket · CIFAR-10, CIFAR-100, SVHN, and Fashion-MNIST | classification accuracy: ($86.58\%$, $86.58\%$, $89.26\%$) sparsity | No clear effect |
| Prune simCLR pre-trained weights and transfer the resulting mask | ResNet-50 ticket · CIFAR-10, CIFAR-100, SVHN, and Fashion-MNIST | classification accuracy: ($91.41\%$, $86.58\%$, $96.48\%$, $89.26\%$) sparsity | No clear effect |
| Prune MoCo pre-trained weights and transfer the resulting mask | ResNet-50 ticket · Pascal VOC 2007 detection and Pascal VOC 2012 segmentation | task performance: ($73.79\%$, $83.22\%$) sparsity | No clear effect |
| Use self-supervised pre-training rather than ImageNet supervised pre-training | ResNet-50 tickets · Natural-image classification at extreme sparsity | generalization | Helped |
| Use supervised ImageNet pre-training rather than self-supervised pre-training | ResNet-50 tickets · VisDA2017 | classification performance: 0.00\% to 89.26\% sparsity | Helped |
| Use MoCo pre-training rather than ImageNet or simCLR pre-training | ResNet-50 tickets · Pascal VOC detection and segmentation | task performance | Helped |
| Use pre-trained initialization instead of random initialization or 5% rewinding | Downstream task tickets · Classification downstream tasks | performance | Helped |
| Use simCLR pre-trained ResNet-152 instead of ResNet-50 to find subnetworks (ablation) | simCLR ticket · CIFAR-100 | accuracy | Helped |
| Use simCLR temperature 0.5 rather than temperature 20.0 (ablation) | simCLR ticket · CIFAR-100 | accuracy: 80.18 -> 82.22 | Helped |
| Use moderate simCLR temperatures from 0.1 to 0.5 rather than extreme temperature 20.0 (ablation) | simCLR ticket · CIFAR-100 | accuracy: 81.81, 81.91, 82.22 -> 80.18 | Helped |
| Use simCLR temperature 1.0 rather than temperature 0.5 (ablation) | simCLR ticket · CIFAR-100 | accuracy: 82.22 -> 81.24 | Hurt |
| Use simCLR temperature 2.0 rather than temperature 0.5 (ablation) | simCLR ticket · CIFAR-100 | accuracy: 82.22 -> 80.76 | Hurt |
Closest papers by meaning
- The Lottery Ticket Hypothesis for Object Recognition (CVPR 2021 · similarity 0.91)
- Data-Efficient Double-Win Lottery Tickets from Robust Pre-training (ICML 2022 · similarity 0.90)
- The Lottery Ticket Hypothesis for Pre-trained BERT Networks (NeurIPS 2020 · similarity 0.90)
- Playing Lottery Tickets with Vision and Language (AAAI 2022 · similarity 0.90)
- Sparse Training from Random Initialization: Aligning Lottery Ticket Masks using Weight Symmetry (ICML 2025 · similarity 0.88)
- Dataset Distillation for Pre-Trained Self-Supervised Vision Models (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.