Research map / Label-efficient and robust learning / OOD and anomaly detection
Aggregation Hides Out-of-Distribution Generalization Failures from Spurious Correlations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes OODSelect, a gradient-based subset selection that finds semantically coherent OOD subsets on which ID and OOD accuracy are negatively correlated (accuracy-on-the-inverse-line), showing aggregate OOD metrics hide spurious-correlation failures.
Problem
Accuracy-on-the-line (strong positive ID-OOD accuracy correlation) across models is often taken as evidence that spurious correlations are rare; the paper asks whether aggregating heterogeneous OOD examples hides subsets where higher ID accuracy predicts lower OOD accuracy.
Approach
Treat the OOD subset as a sigmoid-relaxed binary selection vector and optimize it with Adam (plus a squared budget penalty and cosine annealing) to minimize the Pearson correlation between probit ID accuracy and selected-subset OOD accuracy across a large population of trained models, yielding large subsets with accuracy-on-the-inverse-line.
Key results
- On CXR No Finding, the full OOD set has Pearson R of 0.85 while an OODSelect subset of 55,000 examples has R of -0.46 (and -0.93 at 750-1000 samples in the table).
- On Terra Incognita L46, OODSelect subsets of 1000 examples reach R = -0.77 versus 0.89 for random selection at the same size.
- On PACS, a subset of about 8% of the OOD data has correlation -0.33, while the full dataset has 0.81.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Select OOD examples with OODSelect instead of random sampling | Models evaluated on CXR No Finding · MIMIC-CXR, N=5000 | Pearson R between ID and OOD accuracy: 0.86 (0.06) -> -0.98 (0.01) | Mixed |
| Select OOD examples with OODSelect instead of the most misclassified examples | Models evaluated on CXR No Finding · MIMIC-CXR, N=5000 | Pearson R between ID and OOD accuracy: -0.40 (0.15) -> -0.98 (0.01) | Mixed |
| Select OOD examples with OODSelect instead of random sampling | Models evaluated on WILDSCivilComments · OOD group 4, N=5000 | Pearson R between ID and OOD accuracy: 0.95 (0.02) -> -0.98 (0.01) | Mixed |
| Select OOD examples with OODSelect instead of the most misclassified examples | Models evaluated on WILDSCivilComments · OOD group 4, N=5000 | Pearson R between ID and OOD accuracy: 0.00 (0.20) -> -0.98 (0.01) | Mixed |
| Select OOD examples with OODSelect instead of random sampling | Models evaluated on WILDSCamelyon · Hospital 4, N=5000 | Pearson R between ID and OOD accuracy: 0.99 (0.01) -> -0.99 (0.00) | Mixed |
| Select OOD examples with OODSelect instead of the most misclassified examples | Models evaluated on WILDSCamelyon · Hospital 4, N=5000 | Pearson R between ID and OOD accuracy: -0.32 (0.17) -> -0.99 (0.00) | Mixed |
| Select OOD examples with OODSelect instead of random sampling | Models evaluated on WILDSCamelyon · Hospital 5, N=250 | Pearson R between ID and OOD accuracy: 0.75 (0.10) -> -0.91 (0.03) | Mixed |
| Select OOD examples with OODSelect instead of the most misclassified examples | Models evaluated on WILDSCamelyon · Hospital 5, N=250 | Pearson R between ID and OOD accuracy: -0.18 (0.18) -> -0.91 (0.03) | Mixed |
| Select OOD examples with OODSelect instead of random sampling | Models evaluated on TerraIncognita · L46, N=250 | Pearson R between ID and OOD accuracy: 0.90 (0.04) -> -0.91 (0.03) | Mixed |
| Select OOD examples with OODSelect instead of the most misclassified examples | Models evaluated on TerraIncognita · L46, N=250 | Pearson R between ID and OOD accuracy: 0.00 (0.20) -> -0.91 (0.03) | Mixed |
| Select an OOD subset with OODSelect instead of evaluating the aggregate OOD set | Models evaluated on PACS · PACS, OOD subset comprising 8% of dataset | Pearson R between ID and OOD accuracy: 0.81 -> -0.33 | Mixed |
| Select a smaller OOD subset using OODSelect rather than a larger one | Models evaluated on PACS · PACS, 8% versus 60% of dataset | Pearson R between ID and OOD accuracy: -0.33 -> 0.01 | Mixed |
Surprising findings
- Vision-language model zero-shot accuracy also shows strongly positive ID-OODSelect correlations (e.g., 0.94 on CXR), which the authors caution should not be read as VLM robustness to spurious correlations.
- Distance-based selection in CLIP embedding space fails to recover the negative correlations found by OODSelect (R=-0.92 on VLCS at N=10 for OODSelect versus positive for the baseline).
Limitations
- Requires training up to 4200 models per dataset, making the analysis computationally intensive.
- Semantic explanations of selected subsets are difficult for datasets like WILDS-Camelyon without metadata; VLM/LLM-based descriptions did not give consistent results.
- Some datasets (e.g., PACS) yield very small or noisy negative-correlation subsets, and the existence of such subsets varies across datasets.
Experimental setup
Datasets: CheXpert ChestXray8 PadChest VinDr-CXR MIMIC-CXR-JPG WILDS-Camelyon WILDS-CivilComments PACS VLCS Terra Incognita
Benchmarks: PACS VLCS Terra Incognita WILDS-Camelyon WILDS-CivilComments CXR (MIMIC-CXR OOD)
Metrics: Pearson R Spearman rank correlation Jaccard index (normalized) prevalence shift
Baselines: Random selection Most misclassified selection CLIP farthest-distance selection
Builds on: accuracy-on-the-line DomainBed WILDS Adam
Closest papers by meaning
- The Best of Both Worlds: On the Dilemma of Out-of-distribution Detection (NeurIPS 2024 · similarity 0.90)
- Bridging OOD Detection and Generalization: A Graph-Theoretic View (NeurIPS 2024 · similarity 0.89)
- Spurious Feature Diversification Improves Out-of-distribution Generalization (ICLR 2024 · similarity 0.88)
- Towards Robust Out-of-Distribution Generalization Bounds via Sharpness (ICLR 2024 · similarity 0.87)
- LCA-on-the-Line: Benchmarking Out of Distribution Generalization with Class Taxonomies (ICML 2024 · similarity 0.87)
- Expecting The Unexpected: Towards Broad Out-Of-Distribution Detection (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.