Research map / Label-efficient and robust learning / Self-supervised and contrastive
Data-Efficient Contrastive Self-supervised Learning: Most Beneficial Examples for Supervised Learning Contribute the Least
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Select subsets maximizing expected augmentation similarity (SAS) instead of random subsets | SimCLR on CIFAR100, STL10 · CIFAR100, STL10 | downstream linear accuracy vs random subsets: over 3% | Helped |
| Discard 20% of examples (keep SAS subset) for SimCLR training | SimCLR on CIFAR100 · CIFAR100 | downstream accuracy vs full data | No clear effect |
| Discard 40% of examples (keep SAS subset) for SimCLR training | SimCLR on STL10 and TinyImageNet · STL10, TinyImageNet | downstream accuracy vs full data | No clear effect |
| Use SAS subset of 80% with BYOL instead of full data | BYOL on STL10 · STL10 | downstream accuracy vs full data: outperform training on the full data by 2% | Helped |
| Use SAS subsets instead of random subsets | BYOL on STL10 · STL10 | downstream accuracy vs random subsets: more than 3% | Helped |
| Use SAS subsets instead of random subsets | SimSiam on CIFAR100 · CIFAR100 | downstream accuracy | No clear effect |
| Use SAS subsets instead of random subsets | MoCo · CIFAR100 | downstream accuracy vs random subsets: nearly 2% | Helped |
| Use latent classes from 1% labels with CLIP features instead of ground-truth labels (ablation) | SAS latent class approximation · CIFAR100 | downstream accuracy | No clear effect |
| Use k-means clusters of pretrained model representations as latent classes instead of labels (ablation) | SAS latent class approximation · CIFAR100 | downstream accuracy | Mixed |
| Use zero-shot CLIP ImageNet label matching as latent classes instead of STL10 labels (ablation) | SAS latent class approximation · STL10 | downstream accuracy | Helped |
| Use a partially trained (10% epochs) ResNet-50 or pretrained ResNet-10 as proxy model for augmentation similarity instead of fully pretrained ResNet-50 (ablation) | SAS proxy model · CIFAR100 | downstream accuracy | No clear effect |
| Select least forgettable examples (easy for supervised learning) instead of SAS subsets (ablation) | SSL subset selection · CIFAR100 | downstream accuracy | Hurt |
Closest papers by meaning
- Improving Self-Supervised Learning by Characterizing Idealized Representations (NeurIPS 2022 · similarity 0.91)
- Towards the Generalization of Contrastive Self-Supervised Learning (ICLR 2023 · similarity 0.90)
- Self-Supervised Contrastive Learning is Approximately Supervised Contrastive Learning (NeurIPS 2025 · similarity 0.90)
- A theoretical study of inductive biases in contrastive learning (ICLR 2023 · similarity 0.90)
- On the duality between contrastive and non-contrastive self-supervised learning (ICLR 2023 · similarity 0.90)
- An Augmentation-Aware Theory for Self-Supervised Contrastive Learning (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.