Research map / Label-efficient and robust learning / Domain adaptation and generalization
Set Valued Predictions For Robust Domain Generalization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a set-valued domain generalization framework (SET-COVER) that minimizes prediction set size while enforcing per-domain, per-label recall, with VC-based generalization bounds and experiments on synthetic data and four WILDS datasets.
Problem
Domain generalization needs robust worst-case performance across unseen domains, but single-valued predictors can be poor in some domains; set-valued predictors must achieve a target per-label recall in every domain while keeping sets small.
Approach
Formulates set-valued DG as a constrained problem (minimize average set size subject to per-domain, per-label recall ≥ 1-γ), relaxes it with hinge losses, and solves it with a Lagrangian where per-domain multipliers are updated during training; also gives VC-dimension generalization bounds over domains under conditionally Gaussian assumptions.
Key results
- On Camelyon, SET-COVER reaches 0.96 median min-recall with 1.05 median set size and 71% of domains at >=90% recall, versus 1.79 set size for Robust Conformal, which reaches 93%.
- On Amazon, SET-COVER achieves 96% of domains at >=90% recall with median set size 2.43, compared with 99% and 4.68 for Robust Conformal.
- On iWildCam, SET-COVER attains 82% of domains at >=90% recall with median set size 1.01, versus 2.00 set size for Robust Conformal (86%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SET-COVER instead of Robust Conformal | ResNet-18 set prediction · Camelyon | Median Avg Size: 1.79 $\pm$ 0.16 -> 1.05 $\pm$ 0.03 | Helped |
| Use SET-COVER instead of Robust Conformal | ResNet-18 set prediction · Camelyon | Median Min Recall: 0.98 $\pm$ 0.01 -> 0.96 $\pm$ 0.03 | Hurt |
| Use SET-COVER instead of Robust Conformal | ResNet-18 set prediction · FMoW | Median Min Recall: 0.89 $\pm$ 0.01 -> 0.91 $\pm$ 0.02 | Helped |
| Use SET-COVER instead of Robust Conformal | ResNet-18 set prediction · FMoW | Recall ≥ 90% Pctg: 0.43 $\pm$ 0.15 -> 0.72 $\pm$ 0.22 | Helped |
| Use SET-COVER instead of Robust Conformal | ResNet-18 set prediction · iWildCam | Median Avg Size: 2.00 $\pm$ 0.63 -> 1.01 $\pm$ 0.02 | Helped |
| Use SET-COVER instead of Robust Conformal | ResNet-18 set prediction · iWildCam | Median Min Recall: 0.99 $\pm$ 0.01 -> 0.99 $\pm$ 0.00 | No clear effect |
| Use SET-COVER instead of Robust Conformal | Bert embeddings + 2-layer MLP set prediction · Amazon | Median Avg Size: 4.68 $\pm$ 0.28 -> 2.43 $\pm$ 0.09 | Helped |
| Use SET-COVER instead of Robust Conformal | Bert embeddings + 2-layer MLP set prediction · Amazon | Recall ≥ 90% Pctg: 0.99 $\pm$ 0.00 -> 0.96 $\pm$ 0.02 | Hurt |
| Use SET-COVER instead of ERM | ResNet-18 set prediction · FMoW | Median Min Recall: 0.76 $\pm$ 0.06 -> 0.91 $\pm$ 0.02 | Helped |
| Use SET-COVER instead of ERM | ResNet-18 set prediction · Camelyon | Median Avg Size: 1.0 $\pm$ 0 -> 1.05 $\pm$ 0.03 | Hurt |
| Change the first loss term to punish only wrong classifications (ablation) | SET-COVER · Training stability | learning stability | Helped |
| Use SET-COVER instead of Robust Conformal | 2-layer MLP set prediction · Synthetic data | average prediction set size | Helped |
Surprising findings
- Linear set-valued hypotheses in one dimension cannot shatter more than 2|Y| domains, while in dimension greater than one they can shatter any finite number of domains, implying infinite VC dimension without restrictions on domains.
Limitations
- Theoretical finite-VC guarantees require restrictive assumptions (conditionally Gaussian domains with shared covariance structure up to scaling).
- SET-COVER does not consistently achieve the 90% target in all domains (e.g., 71% on Camelyon and 72% on FMoW).
Experimental setup
Datasets: Camelyon17-WILDS FMoW-WILDS iWildCam-WILDS Amazon-WILDS synthetic Gaussian DGP
Benchmarks: Camelyon17-WILDS FMoW-WILDS iWildCam-WILDS Amazon-WILDS synthetic Gaussian DGP
Metrics: min-recall per domain average prediction set size percentage of domains with recall >= 90%
Baselines: ERM Pooling CDFs (TrainC) Pooling CDFs (CVC) Robust Conformal Prediction
Builds on: conformal prediction Quantile Risk Minimization Pooling-CDFs
Closest papers by meaning
- Robust Domain Generalization under Divergent Marginal and Conditional Distributions (NeurIPS 2026 · similarity 0.91)
- Understanding Domain Generalization: A Noise Robustness Perspective (ICLR 2024 · similarity 0.91)
- Sharpness-Aware Model-Agnostic Long-Tailed Domain Generalization (AAAI 2024 · similarity 0.89)
- Lost Domain Generalization Is a Natural Consequence of Lack of Training Domains (AAAI 2024 · similarity 0.89)
- Unknown Domain Inconsistency Minimization for Domain Generalization (ICLR 2024 · similarity 0.88)
- PracticalDG: Perturbation Distillation on Vision-Language Models for Hybrid Domain Generalization (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.