Research map / Probabilistic and scientific modeling / Treatment effects
Not all distributional shifts are equal: Fine-grained robust conformal inference
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes weighted and debiased robust conformal prediction that reweights training data for identifiable covariate shift while bounding the worst-case Y|X shift in an f-divergence ball, applied to ITE sensitivity analysis.
Problem
Prediction intervals under distributional shift are overly conservative when the shift is treated jointly; covariate shift (identifiable) and Y|X shift (not identifiable) need separate treatment.
Approach
Reweight calibration scores by the estimated covariate likelihood ratio and inflate the miscoverage level via the f-divergence robust function g_{f,rho}, so only the conditional Y|X shift is handled by worst-case robustness; a doubly robust variant (D-WRCP) combines weight estimation with conditional CDF estimation.
Key results
- WRCP and D-WRCP achieve near-exact coverage at the true rho*=0.01 in simulations where CP and WCP undercover and RCP yields infinite-length intervals.
- On the NSLM semi-synthetic task, CP and WCP fail the 80% target while RCP is overly conservative; WRCP and D-WRCP reach approximate coverage and are much shorter than RCP.
- On ACS income and COVID studies, the proposed methods give valid coverage over a range of rho with shorter sets/intervals than RCP.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use WRCP or D-WRCP instead of CP or WCP | Conformal prediction under simulated distribution shift · Simulation | coverage | Helped |
| Use WRCP or D-WRCP instead of RCP | Conformal prediction under simulated distribution shift · Simulation | prediction interval length | Helped |
| Use WRCP or D-WRCP instead of CP or WCP | Conformal prediction for predicting Y(1) in the control group · NSLM | coverage | Helped |
| Use WRCP or D-WRCP instead of RCP | Conformal prediction for predicting Y(1) in the control group · NSLM | prediction interval length | Helped |
| Use WCP instead of CP | Conformal prediction under covariate and conditional shift · ACS income | coverage | Helped |
| Use WRCP or D-WRCP instead of CP or WCP | Conformal prediction under covariate and conditional shift · ACS income | coverage | Helped |
| Use WRCP or D-WRCP instead of RCP | Conformal prediction under covariate and conditional shift · ACS income | prediction set cardinality | Helped |
| Use WCP instead of CP | Conformal prediction for willingness-to-share ratings · COVID information studies | coverage | Helped |
| Use WRCP or D-WRCP instead of CP or WCP | Conformal prediction for willingness-to-share ratings · COVID information studies | coverage | Helped |
| Use WRCP or D-WRCP instead of RCP | Conformal prediction for willingness-to-share ratings · COVID information studies | prediction interval length | Helped |
Surprising findings
- Standard CP and weighted CP undercover under the simulated joint shift, and the joint-robust RCP is so conservative that its intervals are often infinite.
Limitations
- Requires an upper bound rho on the Y|X f-divergence shift, which is hard to specify without target labels.
- Coverage guarantees are approximate and depend on the accuracy of the estimated covariate likelihood ratio.
- Doubly robust variant needs estimation of conditional CDFs, which may be inaccurate in high dimensions.
Experimental setup
Datasets: National Study of Learning Mindsets ACS income dataset COVID information studies
Benchmarks: simulations with X in R^50 NSLM semi-synthetic ACS income (NY to SD) COVID information studies
Metrics: coverage prediction interval length prediction set cardinality
Baselines: CP WCP RCP
Builds on: split conformal prediction weighted conformal prediction robust conformal prediction f-sensitivity model
Closest papers by meaning
- Quantifying Uncertainty in the Presence of Distribution Shifts (NeurIPS 2025 · similarity 0.90)
- Conformal Prediction under Lévy-Prokhorov Distribution Shifts: Robustness to Local and Global Perturbations (NeurIPS 2025 · similarity 0.89)
- Non-exchangeable Conformal Prediction with Optimal Transport: Tackling Distribution Shift with Unlabeled Data (NeurIPS 2025 · similarity 0.88)
- Probabilistic Conformal Prediction with Approximate Conditional Validity (ICLR 2025 · similarity 0.87)
- Rectifying Conformity Scores for Better Conditional Coverage (ICML 2025 · similarity 0.87)
- Generative Conformal Prediction with Optimized Coverage Allocation (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.