Research map / Label-efficient and robust learning / Semi-supervised learning
Complementary Benefits of Contrastive Learning and Self-Training Under Distribution Shift
Saurabh Garg, Amrith Setlur, Zachary Chase Lipton, Sivaraman Balakrishnan, Virginia Smith, Aditi Raghunathan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add self-training (FixMatch) on top of contrastive pretraining (STOC vs CL) | CL · UDA average | accuracy: 62.39 -> 67.59 | Improved |
| Add contrastive pretraining under self-training (STOC vs ST) | ST · UDA average | accuracy: 56.98 -> 67.59 | Improved |
| Add self-training to ERM | ERM · UDA average | accuracy: 48.90 -> 56.98 | Improved |
| Add contrastive pretraining to ERM | ERM · UDA average | accuracy: 48.90 -> 62.39 | Improved |
| Self-training vs contrastive learning on Entity13 | ST vs CL · UDA Entity13 | accuracy: 77.93 -> 76.58 | Mixed |
| Add STOC over CL on Visda | CL · UDA Visda (2 tgts) | accuracy: 63.49 -> 70.08 | Improved |
| Add STOC over CL on CIFAR->CINIC | CL · UDA CIFAR->CINIC | accuracy: 77.51 -> 79.94 | Improved |
| Add self-training over contrastive pretraining in SSL | CL · SSL average | accuracy: 79.27 -> 80.06 | Improved |
| Add self-training over CL in SSL on Visda | CL · SSL Visda | accuracy: 97.67 -> 97.70 | No clear effect |
| Contrastive pretraining on source unlabeled data only instead of source+target (ablation) | CL · UDA Visda (2 tgts) | accuracy: 63.2 -> 33.3 | Hurt |
| Self-training over source-only CL vs standalone CL (ablation) | STOC (source only) · UDA Living17 | accuracy: 67.3 -> 75.0 | Mixed |
| Add self-training on top of CL on target linear probe | CL · UDA linear probe | accuracy | No clear effect |
Closest papers by meaning
- ArCL: Enhancing Contrastive Learning with Augmentation-Robust Representations (ICLR 2023 · similarity 0.89)
- A theoretical study of inductive biases in contrastive learning (ICLR 2023 · similarity 0.88)
- On the duality between contrastive and non-contrastive self-supervised learning (ICLR 2023 · similarity 0.88)
- An Augmentation-Aware Theory for Self-Supervised Contrastive Learning (ICML 2025 · similarity 0.87)
- Multi-modal contrastive learning adapts to intrinsic dimensions of shared latent variables (NeurIPS 2025 · similarity 0.86)
- Self-Supervised Contrastive Learning is Approximately Supervised Contrastive Learning (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
