Research map / Label-efficient and robust learning / OOD and anomaly detection
Coverage-Guaranteed Prediction Sets for Out-of-Distribution Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows split conformal prediction loses marginal coverage under OOD shift and proposes an f-divergence-ball-based corrected SCP (OOD-SCP) with finite-sample coverage guarantees, validated on simulated Gaussian regression data.
Problem
Split conformal prediction assumes exchangeability, which is violated when the unseen target domain differs from source domains, so marginal coverage fails under OOD shift.
Approach
Model the target domain as lying within an f-divergence ball around the convex hull of source domains, derive the worst-case score quantile over this set in closed form via a function g_{f,rho}, and use the resulting threshold (with finite-sample correction for empirical source CDFs) to get guaranteed coverage; extends a single-source result to the multi-source case.
Key results
- Standard SCP falls below the desired 1-alpha coverage under target shift across alpha in {0.05,...,0.3} in 1000-run simulations.
- OOD-SCP keeps coverage above the desired line for all tested alpha in both single-source and multi-source simulated settings.
- The paper gives closed-form worst-case quantile expressions for chi-square and total variation divergences and a binary-search procedure for KL.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use OOD-SCP instead of standard SCP under a single-source distribution shift | Confidence set prediction · Simulated single-source OOD regression | coverage | Helped |
| Apply standard SCP under a source-to-target distribution shift | Confidence set prediction · Simulated single-source OOD regression | coverage | Hurt |
| Use OOD-SCP instead of standard SCP in a multi-source setting | Confidence set prediction · Simulated multi-source OOD regression | coverage | Helped |
Surprising findings
- Trivially applying SCP fails to maintain marginal coverage even under slight distributional shift.
- Mixing source domains into a single mixture domain can leave some source domain outside the coverage guarantee under KL divergence unless rho is large.
Limitations
- Experiments only on simulated linear-Gaussian data; no real OOD datasets because rho cannot be set for existing benchmarks.
- Guarantee holds only when target lies in the f-divergence ball; choosing rho is unresolved.
- Requires known or estimated source score distributions with finite-sample correction.
Experimental setup
Datasets: simulated Gaussian regression data
Benchmarks: simulated OOD regression (single-source and multi-source)
Metrics: marginal coverage prediction set length
Baselines: standard split conformal prediction
Builds on: split conformal prediction conformal prediction under covariate shift (f-divergence single-source)
Closest papers by meaning
- False Coverage Proportion Control for Conformal Prediction (ICML 2025 · similarity 0.90)
- Rectifying Conformity Scores for Better Conditional Coverage (ICML 2025 · similarity 0.89)
- Robust Conformal Prediction Using Privileged Information (NeurIPS 2024 · similarity 0.88)
- Backward Conformal Prediction (NeurIPS 2025 · similarity 0.88)
- Valid Selection among Conformal Sets (NeurIPS 2025 · similarity 0.87)
- Kandinsky Conformal Prediction: Beyond Class- and Covariate-Conditional Coverage (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.