Research map / Label-efficient and robust learning / Recognition and long-tail
Robust Conformal Prediction Using Privileged Information
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes privileged conformal prediction (PCP), a weighted conformal method that uses privileged information explaining data corruption to produce prediction sets with valid marginal coverage despite corrupted training labels or features.
Problem
Conformal prediction assumes i.i.d. training and test data, but training data corrupted by missing or noisy variables induces distribution shift, breaking coverage guarantees for clean test responses.
Approach
Assumes privileged information Z (available only at training) explains corruption indicator M via (X(0),Y(0)) independent of M given Z, reformulates weighted conformal prediction so the test weight is replaced by a fixed calibration-based threshold (aggregating WCP thresholds over calibration points), avoiding the need for test-time Z; also a leave-one-out variant (LOO-PCP) for small data.
Key results
- On the missing response experiment, PCP achieves about 90% coverage on all six datasets, while Two-Staged is conservative at roughly 95% coverage.
- On CIFAR-10N, PCP reaches coverage close to the 90% target with set sizes similar to the infeasible WCP, while naive CP on only clean samples undercovers.
- On IHDP, PCP matches the coverage of infeasible JAW and produces intervals of roughly the same width, while naive jackknife+ and naive WCP undercover.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LOO-PCP instead of naive jackknife+ or Naive WCP | IHDP calibration · IHDP | coverage | Helped |
| Use LOO-PCP without test privileged information instead of Infeasible WCP | IHDP calibration · IHDP | interval length | No clear effect |
| Use PCP instead of Naive CP | missing-response calibration · Missing response datasets | coverage | Helped |
| Use PCP instead of Two-Staged | missing-response calibration · Missing response datasets | interval length | Helped |
| Use PCP without test privileged information instead of Infeasible WCP | missing-response calibration · Missing response datasets | interval length | No clear effect |
| Use Naive CP with noisy labels instead of PCP | CIFAR-10N calibration · CIFAR-10N | coverage | Mixed |
| Calibrate only on clean samples instead of using PCP | CIFAR-10N calibration · CIFAR-10N | coverage | Hurt |
| Use Two-Staged CP instead of PCP | CIFAR-10N calibration · CIFAR-10N | set size | Hurt |
| Estimate corruption probabilities for PCP rather than use true probabilities | PCP · CIFAR-10N | coverage | Mixed |
| Use PCP without test privileged information instead of Infeasible WCP | CIFAR-10N calibration · CIFAR-10N | set size | No clear effect |
Surprising findings
- Naive CP trained with noisy labels overcovers the clean label on CIFAR-10N, consistent with conservative sets under dispersive label noise.
- Estimated weights in PCP yield conservative rather than nominal coverage on CIFAR-10N, though the method is not guaranteed in that setting.
Limitations
- Theoretical validity requires the true corruption probabilities (likelihood ratio weights); estimated weights lack a formal guarantee.
- Relies on the conditional independence assumption (X(0),Y(0)) independent of M given Z, which may be hard to satisfy with real privileged information.
- LOO-PCP guarantee has a factor of 2 in alpha (1-2alpha coverage).
Experimental setup
Datasets: IHDP Facebook1 Facebook2 Bio House Meps19 Blog CIFAR-10N CIFAR-10
Benchmarks: IHDP Facebook1 Facebook2 Bio House Meps19 Blog CIFAR-10N
Metrics: marginal coverage rate average interval length prediction set size
Baselines: Naive CP Naive WCP Two-Staged CP Infeasible WCP JAW Naive jackknife+
Builds on: conformal prediction weighted conformal prediction jackknife+ JAW CQR homogeneous prediction sets
Closest papers by meaning
- An Information Theoretic Perspective on Conformal Prediction (NeurIPS 2024 · similarity 0.90)
- Conformal Prediction with Learned Features (ICML 2024 · similarity 0.89)
- Backward Conformal Prediction (NeurIPS 2025 · similarity 0.88)
- Valid Selection among Conformal Sets (NeurIPS 2025 · similarity 0.88)
- Robust Conformal Prediction with a Single Binary Certificate (ICLR 2025 · similarity 0.88)
- Coverage-Guaranteed Prediction Sets for Out-of-Distribution Data (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.