Research map / Label-efficient and robust learning / OOD and anomaly detection
Unraveling the Key Components of OOD Generalization via Diversification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Studies diversification methods (DivDis, D-BAT) for OOD generalization and finds they are sensitive to unlabeled data spurious ratio, learning algorithm choice (architecture/pretraining), and that more hypotheses do not help.
Problem
Diversification methods that find multiple diverse hypotheses to handle spurious correlations are reported to work well, but their OOD generalization depends on unidentified design factors; the paper asks which components matter.
Approach
Analyzes diversification as three interacting components: the spurious ratio of the unlabeled data, the learning algorithm's inductive bias (architecture and pretraining), and the number of hypotheses K, showing each can cause large accuracy drops and that they are co-dependent, with supporting theory and synthetic/real experiments.
Key results
- Using the second-best architecture/pretraining choice causes up to 20% absolute accuracy drop in diversification methods on Waterbirds and Office-Home.
- D-BAT and DivDis performance drops up to 30% absolute accuracy when the spurious ratio of unlabeled data moves away from their sweet spot (0 and 0.5 respectively).
- Increasing the number of hypotheses K from 2 to 5 does not close the gap between second-best and best models (e.g., D-BAT and DivDis results in Table with K up to 5).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase D-BAT hypothesis count from K=2 to K=5 (ablation) | D-BAT (ViT-B/16 IN) · Waterbirds-CC | worst-group accuracy: 57.1$_{\pm 3.7}$ -> 57.1$_{\pm 3.7}$ | No clear effect |
| increase DivDis hypothesis count from K=2 to K=3 (ablation) | DivDis (MoCo-v2) · Waterbirds-CC | worst-group accuracy: 49.4$_{\pm 10.3}$ -> 51.7$_{\pm 6.0}$ | No clear effect |
| increase DivDis hypothesis count from K=2 to K=4 (ablation) | DivDis (MoCo-v2) · Waterbirds-CC | worst-group accuracy: 49.4$_{\pm 10.3}$ -> 49.6$_{\pm 8.3}$ | No clear effect |
| increase DivDis hypothesis count from K=2 to K=5 (ablation) | DivDis (MoCo-v2) · Waterbirds-CC | worst-group accuracy: 49.4$_{\pm 10.3}$ -> 48.4$_{\pm 0.9}$ | Hurt |
| increase D-BAT hypothesis count from K=2 to K=3 (ablation) | D-BAT (ViT-MAE) · Office-Home | accuracy: 61.9$_{\pm 0.7}$ -> 62.6$_{\pm 0.1}$ | Helped |
| increase D-BAT hypothesis count from K=2 to K=4 (ablation) | D-BAT (ViT-MAE) · Office-Home | accuracy: 61.9$_{\pm 0.7}$ -> 62.6$_{\pm 0.1}$ | Helped |
| increase D-BAT hypothesis count from K=2 to K=5 (ablation) | D-BAT (ViT-MAE) · Office-Home | accuracy: 61.9$_{\pm 0.7}$ -> 62.6$_{\pm 0.1}$ | Helped |
| increase DivDis hypothesis count from K=2 to K=3 (ablation) | DivDis (ResNet50 IN) · Office-Home | accuracy: 55.9$_{\pm 0.6}$ -> 54.6$_{\pm 0.1}$ | Hurt |
| increase DivDis hypothesis count from K=2 to K=4 (ablation) | DivDis (ResNet50 IN) · Office-Home | accuracy: 55.9$_{\pm 0.6}$ -> 53.6$_{\pm 0.4}$ | Hurt |
| increase DivDis hypothesis count from K=2 to K=5 (ablation) | DivDis (ResNet50 IN) · Office-Home | accuracy: 55.9$_{\pm 0.6}$ -> 53.1$_{\pm 0.2}$ | Hurt |
| use unlabeled data with the method-specific optimal spurious ratio | D-BAT and DivDis(-Seq) · M/C and M/F | test accuracy: up to 30\% absolute accuracy drop | Helped |
| choose a second-best rather than the best architecture and pretraining | DivDis and D-BAT · Waterbirds-CC and Office-Home | accuracy: up to 20\% absolute | Hurt |
Surprising findings
- ImageNet performance is not a reliable proxy for choosing the architecture/pretraining for diversification; the best ImageNet model (ViT-MAE) underperforms significantly on the tested datasets.
- Diversification alone can provably fail: there exist 2|D_ood|-1 pairwise diverse hypotheses with random accuracy on OOD data.
Limitations
- Focuses on the spurious ratio as the only characterization of OOD data; other data properties are not studied.
- Mainly image data experiments; conclusions expected but not verified to be data-agnostic.
- Co-dependence experiment performed only with D-BAT, not DivDis.
Experimental setup
Datasets: Waterbirds-CC Office-Home MNIST CIFAR-10 Fashion-MNIST CelebA-CC ImageNet-1k
Benchmarks: Waterbirds-CC Office-Home M/C M/F CelebA-CC synthetic 2D binary task
Metrics: test accuracy worst-group accuracy
Baselines: DivDis D-BAT DivDis-Seq ERM
Builds on: DivDis D-BAT Task Discovery
Closest papers by meaning
- OOD-Chameleon: Is Algorithm Selection for OOD Generalization Learnable? (ICML 2025 · similarity 0.89)
- Out-Of-Distribution Detection with Diversification (Provably) (NeurIPS 2024 · similarity 0.87)
- Bridging OOD Detection and Generalization: A Graph-Theoretic View (NeurIPS 2024 · similarity 0.87)
- Aggregation Hides Out-of-Distribution Generalization Failures from Spurious Correlations (NeurIPS 2025 · similarity 0.87)
- Spurious Feature Diversification Improves Out-of-distribution Generalization (ICLR 2024 · similarity 0.86)
- LCA-on-the-Line: Benchmarking Out of Distribution Generalization with Class Taxonomies (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.