Research map / Label-efficient and robust learning / Recognition and long-tail
Multi-model Ensemble Conformal Prediction in Dynamic Environments
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SAMOCP, a strongly adaptive online conformal prediction method that selects, on the fly, among multiple candidate models (each with its own miss-coverage update) within expert intervals, achieving strongly adaptive regret and valid coverage.
Problem
Conformal prediction under distribution shift in dynamic environments: the efficiency (prediction set size) of adaptive conformal methods depends heavily on which single learning model is used, and no single fixed model performs consistently well.
Approach
Combine multi-model ensembles with strongly adaptive expert aggregation: each expert (an MOCP instance with a bounded lifetime) runs several models each with its own pinball-loss OGD miss-coverage update, and a weighted learner selects among active experts' models, so the prediction set uses whichever model currently performs best.
Key results
- On CIFAR-100C gradual shift (target 90%), SAMOCP achieves average width 5.43 and avg regret 0.92e-3, versus best baseline widths around 5.77 (FACI with DenseNet-121) and coverage 88.16%.
- On CIFAR-10C sudden shift with DenseNet-121, SAMOCP reaches average width 1.24 and single-width 0.69, compared with 1.30 for FACI.
- SAMOCP's per-step complexity is O(T log T) overall, with max active experts g*floor(log2 t), and regret bound of order T^{2/3} V^{1/3} under dynamic regret.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SAMOCP instead of the strongest single-model baseline | Online conformal prediction · CIFAR-100C gradual shift | Avg Width: 5.77 $\pm$ 0.62 -> 5.43 $\pm$ 0.28 | Helped |
| Use SAMOCP instead of the strongest single-model baseline | Online conformal prediction · CIFAR-100C gradual shift | Avg Regret ($\times 10^{-3}$): 1.10 $\pm$ 0.78 -> 0.92 $\pm$ 0.07 | Helped |
| Use SAMOCP instead of the strongest single-model baseline | Online conformal prediction · CIFAR-100C gradual shift | Single Width: 0.28 $\pm$ 0.01 -> 0.29 $\pm$ 0.01 | Helped |
| Use SAMOCP instead of the best-coverage baseline | Online conformal prediction · CIFAR-100C gradual shift | Coverage (%): 89.97 $\pm$ 0.02 -> 88.16 $\pm$ 0.18 | Hurt |
| Use SAMOCP instead of the fastest baseline | Online conformal prediction · CIFAR-100C gradual shift | Run Time: 3.02 $\pm$ 0.06 -> 34.87 $\pm$ 0.67 | Hurt |
| Use SAMOCP instead of the strongest single-model baseline | Online conformal prediction · CIFAR-10C sudden shift | Avg Width: 1.30 $\pm$ 0.12 -> 1.24 $\pm$ 0.06 | Helped |
| Use SAMOCP instead of the strongest single-model baseline | Online conformal prediction · CIFAR-10C sudden shift | Avg Regret ($\times 10^{-3}$): 1.35 $\pm$ 0.93 -> 0.98 $\pm$ 0.11 | Helped |
| Use SAMOCP instead of the strongest single-model baseline | Online conformal prediction · CIFAR-10C sudden shift | Single Width: 0.68 $\pm$ 0.05 -> 0.69 $\pm$ 0.03 | Helped |
| Use SAMOCP instead of the best-coverage baseline | Online conformal prediction · CIFAR-10C sudden shift | Coverage (%): 89.99 $\pm$ 0.01 -> 88.37 $\pm$ 0.23 | Hurt |
| Use SAMOCP instead of the fastest baseline | Online conformal prediction · CIFAR-10C sudden shift | Run Time: 2.89 $\pm$ 0.04 -> 33.75 $\pm$ 0.34 | Hurt |
Surprising findings
- Coverage of SAMOCP (88.16%) and SAOCP(MM) (85.89%) falls below the 90% target on CIFAR-100C, while FACI and ScaleFreeOGD stay closer to 90%, showing a trade-off with efficiency.
Limitations
- Coverage can fall somewhat below the target (e.g., 85.89-88.16% vs 90%) in some settings.
- Runtime is higher than single-model baselines such as FACI and ScaleFreeOGD.
- Experiments are limited to image classification with corrupted CIFAR and TinyImageNet data.
Experimental setup
Datasets: CIFAR-10C CIFAR-100C TinyImageNet-C synthetic transformed images
Benchmarks: CIFAR-10C CIFAR-100C TinyImageNet-C synthetic data
Metrics: coverage average prediction set width average adaptive regret run time single-width ratio
Baselines: FACI ScaleFreeOGD SAOCP SAOCP(MM)
Builds on: SAOCP FACI ScaleFreeOGD conformal prediction strongly adaptive regret
Closest papers by meaning
- Conformal Prediction for Ensembles: Improving Efficiency via Score-Based Aggregation (NeurIPS 2025 · similarity 0.90)
- Optimal transport-based conformal prediction (ICML 2025 · similarity 0.89)
- CBMA: Improving Conformal Prediction through Bayesian Model Averaging (ICLR 2025 · similarity 0.89)
- Generative Conformal Prediction with Optimized Coverage Allocation (NeurIPS 2026 · similarity 0.89)
- Co-optimization for Adaptive Conformal Prediction (NeurIPS 2026 · similarity 0.88)
- Valid Selection among Conformal Sets (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.