Research map / Learning theory and algorithms / Prediction and decision losses
One Sample is Enough to Make Conformal Prediction Robust
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that conformal prediction with a single noise-augmented forward pass, calibrated with a conservative nominal coverage derived from a binary smoothing certificate, yields robust prediction sets (RCP1) comparable to multi-sample smoothing methods at a fraction of the compute.
Problem
Smoothing-based robust conformal prediction (RCP) guarantees coverage under worst-case input perturbations, but needs many Monte-Carlo forward passes per input, which is computationally expensive, especially for large models.
Approach
Vanilla conformal prediction computed on noise-augmented inputs is already robust; by certifying the conformal procedure itself with any binary smoothing certificate (rather than per-score statistics), one sample per input suffices, with the nominal coverage set to 1-alpha' such that the certified lower bound stays above 1-alpha.
Key results
- On ImageNet, RCP1 with one sample attains set sizes similar to BinCP using roughly 64-128 Monte-Carlo samples per input.
- Estimated runtime for RCP1 on 1000 ImageNet inputs with ViT is about 33 seconds versus about 35 minutes for 64 BinCP samples on H-100.
- RCP1 extends smoothing-based robust conformal risk control to segmentation on Cityscapes, with robust false negative rate kept under the risk level 0.15 (robust risk 0.083 for car).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use APS score instead of TPS (ablation) | RCP1 on CIFAR-10 · CIFAR-10, radius 0.06, coverage 0.90 | average set size: 2.70 ± 0.03 -> 2.98 ± 0.02 | Hurt |
| Use APS score instead of TPS (ablation) | RCP1 on CIFAR-10 · CIFAR-10, radius 0.06, coverage 0.90 | empirical coverage: 0.92 ± 0.00 -> 0.92 ± 0.00 | No clear effect |
| Use APS score instead of TPS (ablation) | RCP1 on CIFAR-10 · CIFAR-10, radius 0.25, coverage 0.90 | average set size: 3.70 ± 0.02 -> 3.89 ± 0.03 | Hurt |
| Use APS score instead of TPS (ablation) | RCP1 on CIFAR-10 · CIFAR-10, radius 0.25, coverage 0.90 | empirical coverage: 0.97 ± 0.00 -> 0.97 ± 0.00 | No clear effect |
| Increase perturbation radius from 0.06 to 0.25 | RCP1 TPS · CIFAR-10, coverage 0.90 | average set size: 2.70 ± 0.03 -> 3.70 ± 0.02 | Hurt |
| Increase perturbation radius from 0.06 to 0.25 | RCP1 TPS · CIFAR-10, coverage 0.90 | empirical coverage: 0.92 ± 0.00 -> 0.97 ± 0.00 | Helped |
| Apply robust risk control rather than non-robust risk control | Cityscapes conformal risk control, Pedestrian · Cityscapes, Pedestrian | risk: 0.1474 ± 0.2797 -> 0.1111 ± 0.2588 | Helped |
| Apply robust risk control rather than non-robust risk control | Cityscapes conformal risk control, Car · Cityscapes, Car | risk: 0.1466 ± 0.2582 -> 0.0833 ± 0.2032 | Helped |
| Use APS score instead of TPS (ablation) | RCP1 on CIFAR-10 · CIFAR-10, radius 0.75, coverage 0.90 | average set size: 10.00 ± 0.00 -> 10.00 ± 0.00 | No clear effect |
Surprising findings
- Plain conformal prediction on noise-augmented inputs already shows substantial robustness to adversarial perturbations, before any certificate is applied.
- RCP1 prediction sets are considerably more random across noise draws than BinCP sets, a stated limitation.
Limitations
- Prediction sets are random functions of the smoothing noise, giving considerably higher variance than BinCP.
- Conservative nominal coverage (1-alpha') makes the method less tight than methods using full score distributions; it does not exploit score distribution information.
- Does not outperform BinCP at large sample rates; it targets the compute-limited regime.
Experimental setup
Datasets: CIFAR-10 ImageNet Cityscapes
Benchmarks: CIFAR-10 ImageNet Cityscapes
Metrics: average set size empirical coverage proportion of sets with size less than 5 runtime false negative rate risk
Baselines: BinCP CAS RSCP vanilla conformal prediction Lipschitz/verifier-based RCP
Builds on: BinCP RSCP conformal prediction randomized smoothing PGDSmooth
Closest papers by meaning
- Robust Conformal Prediction with a Single Binary Certificate (ICLR 2025 · similarity 0.94)
- Provably Robust Conformal Prediction with Improved Efficiency (ICLR 2024 · similarity 0.92)
- Robust Yet Efficient Conformal Prediction Sets (ICML 2024 · similarity 0.91)
- Verifiably Robust Conformal Prediction (NeurIPS 2024 · similarity 0.90)
- Exploring the Noise Robustness of Online Conformal Prediction (NeurIPS 2025 · similarity 0.89)
- Efficient Robust Conformal Prediction via Lipschitz-Bounded Networks (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.