Research map / Learning theory and algorithms / Prediction and decision losses
Wasserstein-Regularized Conformal Prediction under General Distribution Shift
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes WR-CP, which bounds the conformal coverage gap under joint distribution shift by a Wasserstein distance between conformal score distributions, splits it into covariate- and concept-shift terms, and trains an MLP with importance weighting plus a Wasserstein regularizer for multi-source conformal prediction.
Problem
Conformal prediction coverage fails under general joint distribution shift (covariate plus concept shift); prior total-variation-based bounds cannot track coverage gap changes across alpha, and worst-case methods yield overly large prediction sets.
Approach
Upper-bound the coverage gap with the 1-Wasserstein distance between calibration and test conformal score distributions, decompose it via pushforward measures into covariate-shift (fixed by importance weighting) and concept-shift (minimized by a regularizer during training) terms, and optimize the model with an empirical Wasserstein penalty.
Key results
- WR-CP reduces average coverage gap to 3.2% across alpha from 0.1 to 0.9, per the abstract.
- WR-CP produces prediction sets about 37% smaller than the worst-case approach (WC-CP) on average, per the abstract.
- Wasserstein distance achieves the highest Spearman correlation with average coverage gap on most of the six datasets (e.g., 0.90 on PeMSD8), versus TV distance.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Wasserstein distance instead of TV distance as the coverage-gap indicator | Conformal prediction under distribution shift · Airfoil self-noise, PeMSD8, Seattle-loop, U.S.-States, Japan-Prefectures | Spearman coefficient with average coverage gap: 0.45 (0.16) -> 0.59 (0.24); 0.86 (0.06) -> 0.90 (0.03); 0.75 (0.09) -> 0.84 (0.05); 0.67 (0.10) -> 0.77 (0.06); 0.37 (0. | Helped |
| use Wasserstein distance instead of TV distance as the coverage-gap indicator | Conformal prediction under distribution shift · PeMSD4 | Spearman coefficient with average coverage gap: 0.88 (0.03) -> 0.84 (0.03) | Hurt |
| add Wasserstein regularization to IW-CP | WR-CP · Six datasets | Normalized Wasserstein distance between calibration and test conformal scores | Helped |
| add importance weighting | Conformal prediction under joint distribution shift · Airfoil self-noise, PeMSD4, PeMSD8, U.S.-States, Japan-Prefectures | Wasserstein distance | Helped |
| add importance weighting | Conformal prediction under joint distribution shift · Seattle-loop | Wasserstein distance | Hurt |
| add importance weighting during training and prediction (ablation) | WR-CP · Six datasets, α=0.2 | Prediction set size: 28.0% | Helped |
| add importance weighting during training and prediction (ablation) | WR-CP · Six datasets, α=0.2 | Average coverage gap: 3.1% vs 2.3% | No clear effect |
| vary β to balance Wasserstein regularization against empirical risk (ablation) | WR-CP · Six datasets, α=0.1 to 0.9 | Coverage gap and prediction set size | Mixed |
| set β=0, removing Wasserstein regularization (ablation) | WR-CP · Multi-source conformal prediction | Coverage gap and prediction set size | Mixed |
Surprising findings
- Importance weighting can increase the Wasserstein distance on the Seattle-loop dataset, causing IW-CP to have worse coverage than vanilla CP there.
- Total variation distance can decrease while Wasserstein distance between CDFs increases, as shown in a toy example.
Limitations
- Theoretical bound relies on assumptions such as a bounded Lebesgue density of the conformal score distribution and asymptotic approximation of the calibration CDF at tau.
- Empirical evaluation is limited to regression tasks with MLP models on six datasets, some of which use synthetic shifts.
Experimental setup
Datasets: airfoil self-noise Seattle-loop PeMSD4 PeMSD8 U.S.-States Japan-Prefectures
Benchmarks: airfoil self-noise PeMSD4 PeMSD8 Seattle-loop U.S.-States Japan-Prefectures
Metrics: coverage gap prediction set size Spearman correlation normalized Wasserstein distance
Baselines: vanilla conformal prediction IW-CP CQR WC-CP total variation distance bound KL divergence
Builds on: split conformal prediction importance-weighted conformal prediction conformalized quantile regression
Closest papers by meaning
- Kandinsky Conformal Prediction: Beyond Class- and Covariate-Conditional Coverage (ICML 2025 · similarity 0.89)
- Backward Conformal Prediction (NeurIPS 2025 · similarity 0.88)
- Robust Conditional Conformal Prediction via Branched Normalizing Flow (NeurIPS 2026 · similarity 0.88)
- Conformal Prediction under Lévy-Prokhorov Distribution Shifts: Robustness to Local and Global Perturbations (NeurIPS 2025 · similarity 0.88)
- Probabilistic Conformal Prediction with Approximate Conditional Validity (ICLR 2025 · similarity 0.87)
- A Unified Comparative Study with Generalized Conformity Scores for Multi-Output Conformal Regression (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.