Research map / Label-efficient and robust learning / Noisy labels
Exploring the Noise Robustness of Online Conformal Prediction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows uniform label noise biases online conformal prediction (ACI) coverage and proposes a robust pinball loss, an unbiased estimate of the clean pinball loss using all-class scores, that restores precise coverage with O(T^-1/2) error rates.
Problem
Online conformal prediction assumes clean labels, but uniform label noise with known rate biases threshold updates, causing persistent coverage gaps (over- or under-coverage) and inefficient prediction sets.
Approach
Replace the pinball loss on noisy scores with a weighted combination: (1/(1-eps)) pinball loss on the noisy score minus (eps/(K(1-eps))) times the sum of pinball losses over all K class scores. This is unbiased for the clean pinball loss in expectation, so ACI variants can be updated without ground-truth labels.
Key results
- On ImageNet with alpha=0.1, epsilon=0.15 and dynamic learning rate, ACI with robust pinball loss reduces coverage gap from 8.372% to 0.183% and set size from 171.2 to 13.10.
- Robust pinball loss gives an O(T^-1/2) convergence rate for both empirical and expected coverage error under uniform label noise.
- Across LAC, APS, RAPS and SAPS scores, the robust loss consistently reduces coverage gap and set size relative to the standard pinball loss baseline under noise rates 0.05-0.15.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace standard pinball loss with robust pinball loss | ACI with dynamic learning rate · ImageNet, ε=0.15, α=0.1 | CovGap (%): 8.372 -> 0.183 | Helped |
| Replace standard pinball loss with robust pinball loss | ACI with dynamic learning rate · ImageNet, ε=0.15, α=0.1 | Size: 171.2 -> 13.10 | Helped |
| Replace standard pinball loss with robust pinball loss | ACI with dynamic learning rate · CIFAR100, ε=0.15, α=0.1 | CovGap (%): 8.403 -> 0.214 | Helped |
| Replace standard pinball loss with robust pinball loss | ACI with dynamic learning rate · CIFAR100, ε=0.15, α=0.1 | Size: 29.41 -> 4.065 | Helped |
Surprising findings
- Standard online conformal prediction rarely under-covers under label noise, since under-coverage requires prediction sets of size at least K(1-alpha).
Limitations
- Analysis restricted to uniform label noise with a known noise rate.
- Empirical convergence rate O(T^-1/2) is slower than the O(T^-1) rate of noise-free ACI.
- Relies on bounded non-conformity scores and a known noise rate assumption.
Experimental setup
Datasets: CIFAR-100 ImageNet CIFAR-10
Benchmarks: CIFAR-100 ImageNet
Metrics: coverage gap prediction set size long-run coverage local coverage empirical coverage error expected coverage error
Baselines: ACI with constant learning rate ACI with dynamic learning rate SAOCP standard pinball loss ACI
Builds on: adaptive conformal inference strongly adaptive online conformal prediction regularized adaptive prediction sets
Closest papers by meaning
- Stochastic Online Conformal Prediction with Semi-Bandit Feedback (ICML 2025 · similarity 0.90)
- One Sample is Enough to Make Conformal Prediction Robust (NeurIPS 2025 · similarity 0.89)
- Adaptive Conformal Inference by Betting (ICML 2024 · similarity 0.89)
- Robust Conformal Prediction with a Single Binary Certificate (ICLR 2025 · similarity 0.88)
- Leave-One-Out Stable Conformal Prediction (ICLR 2025 · similarity 0.88)
- Generative Conformal Prediction with Optimized Coverage Allocation (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.