Research map / Learning theory and algorithms / Prediction and decision losses
Online Conformal Prediction via Online Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes online conformal prediction via projected online gradient descent on the quantile loss over arbitrary linear conformal predictors, giving adversarial long-run coverage and time-conditional coverage for stochastic (e.g., AR) data.
Problem
Online conformal prediction typically guarantees only long-run marginal coverage under adversarial data, which says little about coverage of individual confidence sets; the paper seeks guarantees that hold conditionally on the past for stochastic dependent data.
Approach
Treat online conformal thresholds as parametric linear predictors and update their parameters by batched projected online gradient descent on the (1-alpha) quantile loss, which generalizes ACI beyond scalar bang-bang updates and yields conditional coverage when the model is well-specified and mixing-based guarantees under mis-specification.
Key results
- On the pre-registered ERCOT experiment, LQT reduces average set size by roughly 10% or more and quantile loss by over 30% relative to baselines (e.g., MSFT transformer quantile loss 0.115 vs 0.289 for SQT).
- All 78 pre-registered hypothesis tests comparing baselines to LQT(fixed) are rejected at the 0.05 level.
- Across datasets, the advantage of the proposed algorithms grows with first-lag autocorrelation of the conformal scores, and all methods achieve at least 0.88 long-run coverage.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LQT instead of PID(T) | Online conformal prediction with the MSFT Prophet forecaster · MSFT Prophet | quantile loss (avg): 0.134 -> 0.104 | Helped |
| Use LQT instead of PID(T) | Online conformal prediction with the MSFT Prophet forecaster · MSFT Prophet | set size (avg): 2.938 -> 2.718 | Helped |
| Use SQT+AR instead of PID(A) | Online conformal prediction with the MSFT Transformer forecaster · MSFT Transformer | quantile loss (avg): 0.148 -> 0.107 | Helped |
| Use SQT+AR instead of PID(A) | Online conformal prediction with the MSFT Transformer forecaster · MSFT Transformer | set size (avg): 6.104 -> 5.852 | Helped |
| Use LQT instead of PID(T) | Online conformal prediction on Elec2 · Elec2 | set size (avg): 0.201 -> 0.16 | Helped |
| Use LQT instead of PID(T) | Online conformal prediction on ERCOT · ERCOT | quantile loss (avg): 42.344 -> 29.095 | Helped |
| Use LQT instead of PID(T) | Online conformal prediction on ERCOT · ERCOT | set size (avg): 778.808 -> 691.496 | Helped |
| Use a decaying rather than fixed learning rate for LQT (ablation) | LQT on ERCOT · ERCOT | quantile loss (avg): 29.095 -> 29.158 | Hurt |
| Use a decaying rather than fixed learning rate for LQT (ablation) | LQT on ERCOT · ERCOT | set size (avg): 691.496 -> 691.414 | No clear effect |
Surprising findings
- Baselines tuned on the test set still underperform the proposed method when score autocorrelation is strong.
- Minimizing an arbitrary loss such as set size subject to long-run coverage is argued to be generally impossible in the adversarial setting, motivating the quantile loss.
Limitations
- Conditional coverage guarantees require well-specification of linear quantile model or mixing assumptions on the data.
- Theoretical trade-off: learning rates that give fast convergence destroy the adversarial long-run coverage bound.
- Experiments use simple linear AR-type conformal predictors; richer predictors are left to future work.
Experimental setup
Datasets: AMZN stock returns GOOGL stock returns MSFT stock returns Delhi daily climate Elec2 ERCOT electricity load (pre-registered)
Benchmarks: AMZN GOOGL MSFT Delhi daily climate Elec2 ERCOT
Metrics: long-run coverage quantile loss conformal set size runtime hypothesis test p-values
Baselines: ACI SQT PI PID(T) PID(A) SQT+AR
Builds on: Adaptive conformal inference (ACI) conformal PID control ergodic mirror descent
Closest papers by meaning
- The Relationship Between No-Regret Learning and Online Conformal Prediction (ICML 2025 · similarity 0.90)
- Online conformal prediction with decaying step sizes (ICML 2024 · similarity 0.90)
- Backward Conformal Prediction (NeurIPS 2025 · similarity 0.89)
- Adaptive Conformal Inference by Betting (ICML 2024 · similarity 0.89)
- Conformal Validity Guarantees Exist for Any Data Distribution (and How to Find Them) (ICML 2024 · similarity 0.89)
- Stochastic Online Conformal Prediction with Semi-Bandit Feedback (ICML 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.