Research map / Label-efficient and robust learning / Self-supervised and contrastive
Optimal Representations for Covariate Shift
Yangjun Ruan, Yann Dubois, Chris J. Maddison
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune CLIP L with CAD (ablation) | CLIP L · DomainBed benchmarks | accuracy: 80.7 $\pm$ 0.4 -> 81.4 $\pm$ 0.8 (VLCS); 93.7 $\pm$ 0.8 -> 94.7 $\pm$ 0.4 (PACS); 79.9 $\pm$ 0.1 -> 80.2 $\pm$ 0.2 (Offi | Mixed |
| Fine-tune CLIP L on LAION with an entropy bottleneck instead of without one (ablation) | CLIP L · ImageNet distribution-shift datasets | accuracy: 73.8 -> 74.2 (IN); 62.1 -> 62.7 (IN-V2); 37.0 -> 38.9 (IN-S); 56.9 -> 58.1 (YT-BB); 68.8 -> 70.1 (IN-Vid); 41.3 -> 42.1 | Improved |
| Fine-tune CLIP L on LAION without an entropy bottleneck (ablation) | CLIP L · ImageNet distribution-shift datasets | accuracy: 75.2 -> 73.8 (IN); 64.2 -> 62.1 (IN-V2); 41.0 -> 37.0 (IN-S); 58.4 -> 56.9 (YT-BB); 71.6 -> 68.8 (IN-Vid); 42.8 -> 41.3 | Hurt |
| Use domain-agnostic augmentations rather than non-domain-agnostic standard or intra-domain augmentations (ablation) | CAD representation learning · PACS | worst-case log likelihood | Improved |
| Use approximately domain-agnostic augmentations rather than strictly domain-agnostic augmentations (ablation) | CAD representation learning · PACS | worst-case log likelihood | No clear effect |
| Increase the domain bottleneck weight (ablation) | CAD/Ent representation learning · PACS | worst-case target log likelihood | Improved |
| Increase the domain bottleneck weight beyond approximately 10^2 (ablation) | CAD/Ent representation learning · PACS | source performance | Hurt |
| Add a domain bottleneck (ablation) | Representation learning on PACS · PACS | worst-case target log likelihood | Improved |
| Train representations without access to one target domain (ablation) | CAD representation learning · PACS | worst-case log likelihood: -0.8 $\pm$ 0.2 -> -4.2 $\pm$ 0.2 | Hurt |
Closest papers by meaning
- How robust is unsupervised representation learning to distribution shift? (ICLR 2023 · similarity 0.88)
- Bayesian Nonparametrics Meets Data-Driven Distributionally Robust Optimization (NeurIPS 2024 · similarity 0.86)
- Transformation-Invariant Learning and Theoretical Guarantees for OOD Generalization (NeurIPS 2024 · similarity 0.85)
- Not all distributional shifts are equal: Fine-grained robust conformal inference (ICML 2024 · similarity 0.85)
- Quantifying Uncertainty in the Presence of Distribution Shifts (NeurIPS 2025 · similarity 0.85)
- Tolerant Algorithms for Learning with Arbitrary Covariate Shift (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
