Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
CBMA: Improving Conformal Prediction through Bayesian Model Averaging
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CBMA, which averages posterior predictive conformity scores across candidate Bayesian models using posterior model probabilities, giving valid full-conformal prediction sets that converge to the optimal conformal Bayes set when the true model is included.
Problem
Conformal Bayes prediction sets are only efficient when the Bayesian model is correctly specified, and selecting one model in advance is hard under model uncertainty; existing conformal aggregation methods lose coverage, need data splitting, or do not target efficiency.
Approach
Combine per-model posterior predictive conformity scores into one score weighted by posterior model probabilities and test-point predictive densities (a hierarchical Bayesian mixture), which is exchangeable and so yields valid full conformal sets without data splitting, and converges to the true-model conformal Bayes set.
Key results
- On the quadratic model with n=100, CBMA mean interval length 0.535 vs CBM1 0.534 and BMA 0.537, with coverage near 0.80.
- In the Hermite simulation (n=100, theta=1), CBMA mean length 2.634 versus best individual CB 2.689 and BMA 3.122, with coverage 0.792.
- On California Housing, CBMA intervals reach shortest lengths at n=150 while CB and CBMA attain target coverage 0.80; majority-vote aggregated intervals were 1.08-1.15 times longer than CBMA.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| aggregate Bayesian model conformity scores with CBMA instead of using a single-model conformal Bayes interval | Conformal prediction · Hermite polynomials, theta=1, n=100 | mean length: 2.667 -> 2.634 | Helped |
| aggregate Bayesian model conformity scores with CBMA instead of using a single-model conformal Bayes interval | Conformal prediction · Hermite polynomials, theta=1, n=50 | mean length: 2.740 -> 2.683 | Helped |
| aggregate Bayesian model conformity scores with CBMA instead of using a single-model conformal Bayes interval | Conformal prediction · Hermite polynomials, theta=10, n=100 | mean length: 2.722 -> 2.683 | Helped |
| aggregate Bayesian model conformity scores with CBMA instead of using a single-model conformal Bayes interval | Conformal prediction · Quadratic model, n=100 | mean length: 0.534 -> 0.535 | No clear effect |
| aggregate Bayesian model conformity scores with CBMA instead of using BMA prediction | Conformal prediction · Quadratic model, n=200 | mean length: 0.522 -> 0.522 | No clear effect |
| use CBMA conformal prediction rather than BMA prediction | Conformal prediction · Hermite polynomials, theta=1, n=100 | mean length: 3.122 -> 2.634 | Helped |
| use conformal prediction with CBMA rather than Bayesian prediction sets | Bayesian model averaging · Hermite polynomials, theta=1, n=100 | coverage: 0.831 -> 0.792 | Helped |
Surprising findings
- Bayes credible sets over-cover substantially while conformal Bayes sets hit target coverage.
- On the quadratic model, CBMA length is essentially equal to the true-model CB length rather than clearly shorter, indicating little gain when the true model is among candidates and sample size is small.
Limitations
- Accuracy depends on MCMC sampling error, which can affect the constructed sets.
- Optimality guarantee relies on true model being in the candidate set or KL-closest model, and on regularity conditions for posterior model consistency.
- Experiments are small-scale simulations and a single low-dimensional real dataset with 2-covariate models.
Experimental setup
Datasets: simulated quadratic regression data simulated Hermite-polynomial regression data California Housing
Benchmarks: simulated quadratic model Hermite polynomial approximation simulation California Housing
Metrics: empirical coverage mean interval length
Baselines: conformal Bayes (CB) per model Bayes posterior predictive sets BMA majority vote conformal set aggregation
Builds on: full conformal prediction conformal Bayes Bayesian model averaging scalable full conformal Bayes (Add-One-In importance sampling)
Closest papers by meaning
- Conformal Prediction as Bayesian Quadrature (ICML 2025 · similarity 0.91)
- Multi-model Ensemble Conformal Prediction in Dynamic Environments (NeurIPS 2024 · similarity 0.89)
- Optimal transport-based conformal prediction (ICML 2025 · similarity 0.88)
- Conformity Score Averaging for Classification (ICML 2025 · similarity 0.88)
- Reliable Uncertainty Quantification in Machine Learning via Conformal Prediction (AAAI 2025 · similarity 0.88)
- Generative Conformal Prediction with Optimized Coverage Allocation (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.