Research map / Label-efficient and robust learning / Self-supervised and contrastive
Beyond Supervised vs. Unsupervised: Representative Benchmarking and Analysis of Image Representation Learning
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use supervised pretraining rather than unsupervised pretraining | ResNet-50 representations · ImageNet | k-NN accuracy: 73.41 vs. 63.70 | Helped |
| Use Barlow Twins rather than supervised pretraining | ResNet-50 representations · Flowers | k-NN accuracy: 77.96 vs. 86.18 | Helped |
| Use unsupervised pretraining rather than supervised pretraining | ResNet-50 representations · NABirds | k-NN accuracy: 43.25 vs. 22.29 | Hurt |
| Use supervised pretraining rather than unsupervised pretraining | ResNet-50 representations · ImageNet | k-means accuracy: 58.92 vs. 38.30 | Helped |
| Use Barlow Twins rather than supervised pretraining | ResNet-50 representations · Flowers | k-means accuracy: 54.97 vs. 63.70 | Helped |
| Increase k-means cluster count from k=1000 to k=3000 | ImageNet representations · ImageNet | k-means accuracy: +6.74 | Helped |
| Use a clustering head instead of k-means on learned embeddings | DeepCluster v2 · ImageNet, k=3000 | overclustering accuracy: 43.02 vs. 54.35 | Helped |
| Use a clustering head instead of k-means on learned embeddings | SwAV · ImageNet, k=3000 | overclustering accuracy: 37.94 vs. 48.9 | Helped |
| Train SimCLR for more epochs (ablation) | SimCLR · ImageNet | linear CKA similarity | No clear effect |
| Remove crops from DeepCluster training (ablation) | DeepCluster · ImageNet | linear CKA similarity | Hurt |
| Remove crops from SwAV training (ablation) | SwAV · ImageNet | linear CKA similarity | Hurt |
| Reduce SwAV batch size to 256 (ablation) | SwAV · ImageNet | linear CKA similarity | Hurt |
Closest papers by meaning
- Un-mix: Rethinking Image Mixtures for Unsupervised Visual Representation Learning (AAAI 2022 · similarity 0.90)
- Unsupervised Representation Learning by Predicting Image Rotations (ICLR 2018 · similarity 0.90)
- $\mathbb{X}$-Sample Contrastive Loss: Improving Contrastive Learning with Sample Similarity Graphs (ICLR 2025 · similarity 0.88)
- SELECT: A Large-Scale Benchmark of Data Curation Strategies for Image Classification (NeurIPS 2024 · similarity 0.88)
- Towards an Explainable Comparison and Alignment of Feature Embeddings (ICML 2025 · similarity 0.88)
- I-Con: A Unifying Framework for Representation Learning (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.