Research map / Label-efficient and robust learning / Self-supervised and contrastive
DABS 2.0: Improved Datasets and Algorithms for Universal Self-Supervision
Alex Tamkin, Gaurab Banerjee, Mohamed Owda, Vincent Liu, Shashank Rammoorthy, Noah Goodman
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| pretrain with MAE (50% corruption) vs no pretraining | Domain-agnostic Transformer · Satellite Images | average transfer metric: 62.3 -> 86.3 | Improved |
| pretrain with ShED (15%) vs no pretraining | Domain-agnostic Transformer · Particle Physics | average transfer metric: 54.8 -> 68.0 | Improved |
| pretrain with ShED (15%) vs no pretraining | Domain-agnostic Transformer · Wearable Sensors | average transfer metric: 69.8 -> 85.2 | Improved |
| pretrain with ShED (15%) vs no pretraining | Domain-agnostic Transformer · Satellite Images | average transfer metric: 62.3 -> 57.8 | Hurt |
| pretrain with ShED instead of MAE | Domain-agnostic Transformer · Protein Biology | average transfer metric: 32.2 -> 36.8 | Improved |
| raise ShED corruption rate from 15% to 50% (ablation) | ShED · Wearable Sensors | average transfer metric: 85.2 -> 65.2 | Hurt |
| raise ShED corruption rate from 15% to 50% (ablation) | ShED · Multilingual Text | average transfer metric: 57.5 -> 64.0 | Improved |
| raise ShED corruption rate from 15% to 85% (ablation) | ShED · Natural Images | average transfer metric: 16.1 -> 24.6 | Improved |
| raise MAE corruption rate from 15% to 50% (ablation) | MAE · Genomics | average transfer metric: 26.6 -> 39.0 | Improved |
| raise MAE corruption rate from 50% to 85% (ablation) | MAE · Genomics | average transfer metric: 39.0 -> 25.7 | Hurt |
| raise MAE corruption rate from 15% to 85% (ablation) | MAE · Captioned Images | average transfer metric: 51.4 -> 50.0 | No clear effect |
| use Capri (15%) vs no pretraining | Domain-agnostic Transformer · Satellite Images | average transfer metric: 62.3 -> 11.1 | Hurt |
Closest papers by meaning
- DABS: a Domain-Agnostic Benchmark for Self-Supervised Learning (NeurIPS 2021 · similarity 0.87)
- scSSL-Bench: Benchmarking Self-Supervised Learning for Single-Cell Data (ICML 2025 · similarity 0.86)
- The SSL Interplay: Augmentations, Inductive Bias, and Generalization (ICML 2023 · similarity 0.85)
- Self-Guided Masked Autoencoders for Domain-Agnostic Self-Supervised Learning (ICLR 2024 · similarity 0.84)
- Clustering Properties of Self-Supervised Learning (ICML 2025 · similarity 0.83)
- Boost Self-Supervised Dataset Distillation via Parameterization, Predefined Augmentation, and Approximation (ICLR 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
