Research map / Learning theory and algorithms / Prediction and decision losses
Probabilistic Conformal Prediction with Approximate Conditional Validity
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CP² (conformal with conditional predictive distribution), a split-conformal framework that calibrates explicit or implicit conditional distribution estimates to get marginal validity and approximate conditional coverage, with instances CP²-HPD and CP²-PCP.
Problem
Conformal prediction usually gives only marginal coverage, while conditional coverage is impossible without distributional assumptions; PCP uses fixed-radius balls that under- or over-cover under heteroscedasticity and multimodality, and 1D methods struggle with multi-dimensional targets.
Approach
Use a conditional distribution estimate to define a locally adaptive scale tau_{x,z} (smallest scale whose estimated probability reaches 1-alpha), with conformity score lambda as minimal covering radius; the transformed score f_tau^{-1}(lambda) is calibrated by split conformal, giving marginal validity and non-asymptotic approximate conditional bounds in terms of total variation error.
Key results
- On the synthetic multimodal example, CP²-HPD and CQR stay close to 0.9 conditional coverage while PCP varies between 0.85 and 0.95, and CP²-HPD gives shorter sets than CQR and PCP.
- On real regression datasets, the authors report CP²-PCP achieves valid conditional coverage on all datasets while PCP consistently struggles, and CQR fails on blog and temp.
- In multi-target experiments, PCP significantly undercovers on rf1, rf2 and scm1d, while CP² comes close to the nominal 0.9; CQR/CQR2/CHR overcover on scm20d and sgemm.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CP²-HPD instead of PCP | Synthetic 4-Gaussian mixture · Synthetic 4-Gaussian mixture | prediction set length | Helped |
| Use CP²-HPD instead of PCP | Synthetic 4-Gaussian mixture · Synthetic 4-Gaussian mixture | conditional coverage | Helped |
| Use CP²-PCP instead of PCP | Real-world regression · Real-world regression datasets | worst-slab conditional coverage | Helped |
| Use CP²-PCP instead of quantile regression-based methods | Real-world regression · bike, bio, temp | prediction set size | Helped |
| Use CQR | Real-world regression · blog, temp | worst-slab conditional coverage | Hurt |
| Use CHR | Real-world regression · Real-world regression datasets | worst-slab conditional coverage | Hurt |
| Use CP²-PCP instead of PCP | fb1 clustering evaluation · fb1 HDBSCAN clusters and outliers | conditional coverage | Helped |
| Use CP² instead of PCP | Multi-target regression · rf1, rf2, scm1d | worst-slab conditional coverage | Helped |
| Use CP² instead of competing methods | Multi-target regression · indoor, scm1d, sgemm_small | projected set area rank | Helped |
| Use ΠY|X directly with fₜ(v)=t, removing conformal calibration (ablation) | CP²-PCP · Real-world regression datasets | marginal coverage | Hurt |
Surprising findings
- Naive application of 1D methods (CQR, CQR2, CHR) to multiple outputs produces significant overcoverage.
Limitations
- Exact conditional coverage is impossible without distributional assumptions; guarantees depend on the total variation error of the conditional estimate.
- Performance relies on a good conditional distribution estimator; the HPD variant's procedures are non-trivial and only tested on synthetic data.
- Multi-target area comparisons used approximate grid-based computation with fewer samples due to computational constraints.
Experimental setup
Datasets: synthetic 4-component Gaussian mixture bike bio blog fb1 fb2 meps19 meps20 meps21 temp rf1 rf2
Benchmarks: bike bio blog fb1 fb2 meps19 meps20 meps21 temp rf1 rf2 scm1d
Metrics: marginal coverage worst-slab conditional coverage cluster-wise conditional coverage prediction set size (scaled by response std) 2D projection area
Baselines: PCP CQR CQR2 CHR standard split conformal
Builds on: Probabilistic Conformal Prediction Conformalized Quantile Regression Conformalized Histogram Regression
Closest papers by meaning
- Rectifying Conformity Scores for Better Conditional Coverage (ICML 2025 · similarity 0.93)
- Kandinsky Conformal Prediction: Beyond Class- and Covariate-Conditional Coverage (ICML 2025 · similarity 0.91)
- Backward Conformal Prediction (NeurIPS 2025 · similarity 0.91)
- Co-optimization for Adaptive Conformal Prediction (NeurIPS 2026 · similarity 0.90)
- Generative Conformal Prediction with Optimized Coverage Allocation (NeurIPS 2026 · similarity 0.90)
- Valid Selection among Conformal Sets (NeurIPS 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.