Research map / Label-efficient and robust learning / Self-supervised and contrastive
Extending and Analyzing Self-Supervised Learning Across Domains
Bram Wallace, Bharath Hariharan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| pretrain on all unlabeled data rather than only the labeled 10% (ablation) | pretext + linear head · 10% label setting, 16 domains | accuracy: drops an average of approximately 10% | Improved |
| use Rotation instead of random initialization | Rotation · Kather | accuracy: 79% -> <60% | Hurt |
| use Rotation pretext on domains lacking canonical orientation (Biological) | Rotation · Biological group | accuracy | Hurt |
| use Instance Discrimination instead of Rotation on Biological domains | Instance Discrimination · Biological group | accuracy | Improved |
| use pretext + linear head with 10% labels instead of full supervision on 10% labels | Rotation · Internet and Scenes & Textures, 10% labels | accuracy | Improved |
| use Autoencoding pretext with 10% labels | Autoencoder · Biological, 10% labels | accuracy | Improved |
| use 2000 Jigsaw permutations instead of 100 or 10,000 (ablation) | Jigsaw · 16 domains | accuracy | Improved |
| pretext test on unseen classes (pretext generalization) as predictor of downstream accuracy | Rotation · 16 domains | downstream accuracy | Improved |
| relate Instance Discrimination pretext loss to downstream accuracy | Instance Discrimination · 16 domains | downstream accuracy | No clear effect |
| use Jigsaw instead of Rotation/ID for random-label fit | Jigsaw · 16 domains | random vs normal label train accuracy correlation: 0.59 vs 0.46 vs 0.45 | Mixed |
Closest papers by meaning
- Tailoring Self-Supervision for Supervised Learning (ECCV 2022 · similarity 0.88)
- Towards Efficient and Effective Self-Supervised Learning of Visual Representations (ECCV 2022 · similarity 0.88)
- Scaling and Benchmarking Self-Supervised Visual Representation Learning (ICCV 2019 · similarity 0.88)
- When Does Contrastive Visual Representation Learning Work? (CVPR 2022 · similarity 0.88)
- Self-supervised Representation Learning from Random Data Projectors (ICLR 2024 · similarity 0.86)
- Can Generative Models Improve Self-Supervised Representation Learning? (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
