Research map / Learning theory and algorithms / Prediction and decision losses
Conformal Risk Training: End-to-End Optimization of Conformal Risk Control
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Extends conformal risk control to any optimized certainty-equivalent (OCE) risk, including CVaR, and proposes conformal risk training, which differentiates through the CRC calibration step to fine-tune models end-to-end.
Problem
Post-hoc conformal risk control only bounds expected loss and can substantially degrade average-case performance because the model receives no feedback; it also cannot directly control tail risks such as CVaR.
Approach
Show that any OCE risk can be bounded via a monotone transformation of the loss (t + phi(L - t)), giving a finite-sample CORC procedure, and train models end-to-end by differentiating through the risk-controlling threshold lambda(theta) on pseudo-calibration minibatches.
Key results
- On polyp segmentation, conformal risk training reduces FPR by 23-42% on average across FNR targets (0.01, 0.05, 0.1) relative to post-hoc CRC while keeping FNR controlled.
- On battery storage, conformal risk training achieves 7.2% to 22.6% mean improvement in task loss over post-hoc conformal CVaR control across tested alpha and delta.
- Fine-tuning with the task loss alone does not improve average task loss when controlling tail risk.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune PraNet end-to-end with conformal risk training instead of applying CRC post-hoc | PraNet · Tumor image segmentation test set | FPR: reduces FPR on average by 23-42% | Helped |
| Fine-tune PraNet with cross-entropy, then apply CRC | PraNet · Tumor image segmentation test set | FPR | No clear effect |
| Fine-tune PraNet with conformal risk training rather than post-hoc CRC | PraNet · Tumor image segmentation test set | cross-entropy loss | Hurt |
| Fine-tune the price prediction model with conformal risk training rather than post-hoc conformal CVaR control | Battery storage operation price prediction model · Battery storage operations | task loss: between 7.2% and 22.6% mean improvement | Helped |
| Fine-tune the price prediction model using task loss, then apply conformal CVaR control | Battery storage operation price prediction model · Battery storage operations | task loss | No clear effect |
| Perturb t from its tuned value t₀ (ablation) | Conformal CVaR control · Battery storage operations | task loss: within 1 standard deviation | No clear effect |
| Increase the calibration set size (ablation) | Conformal CVaR control · Battery storage operations | task loss | Helped |
Surprising findings
- Fine-tuning with a decision-focused task loss followed by post-hoc CVaR control does not improve task loss over post-hoc control.
- Fine-tuning PraNet with cross-entropy and then applying post-hoc CRC gives no meaningful improvement over post-hoc CRC alone.
Limitations
- Risk guarantee only applies to monotone and exchangeable losses.
- Exact gradient is derived only in certain cases; no full characterization of when it exists.
- Requires a hyperparameter t chosen on held-out data; results sensitive to its choice in some settings.
Experimental setup
Datasets: CVC-ClinicDB CVC-ColonDB ETIS-LaribPolypDB Kvasir-SEG grid-scale battery storage electricity price data
Benchmarks: colonoscopy polyp segmentation (4 public datasets) battery storage operation task
Metrics: false negative rate false positive rate CVaR of financial risk task loss / profit
Baselines: post-hoc CRC cross-entropy fine-tuning fine-tune task loss post-hoc conformal CVaR control
Builds on: conformal risk control conformal training decision-focused learning PraNet
Closest papers by meaning
- Conformal Risk Control (ICLR 2024 · similarity 0.90)
- Non-Exchangeable Conformal Risk Control (ICLR 2024 · similarity 0.87)
- Verifiably Robust Conformal Prediction (NeurIPS 2024 · similarity 0.87)
- Class Adaptive Conformal Training (NeurIPS 2026 · similarity 0.86)
- The Pitfalls and Promise of Conformal Inference Under Adversarial Attacks (ICML 2024 · similarity 0.85)
- Provably Reliable Conformal Prediction Sets in the Presence of Data Poisoning (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.