Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Credal Wrapper of Model Averaging for Uncertainty Estimation in Classification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes the credal wrapper, which turns the per-class min/max of a few sampled predictive distributions from BNNs or deep ensembles into probability intervals, a credal set, and an intersection probability for prediction, improving OOD detection and calibration on corrupted data.
Problem
Limited number of sampled predictive distributions in BNN model averaging and deep ensembles gives poor epistemic uncertainty estimates and overconfident predictions under corruption and OOD inputs.
Approach
Build per-class probability intervals from the min and max over the finite set of sampled softmax outputs, form the induced credal set, use the intersection probability (equal-trust convex combination of bounds) as the single prediction, and measure EU as upper minus lower entropy; a PIA approximation reduces cost for many classes.
Key results
- On CIFAR10 with VGG16 DEs, the credal wrapper raises SVHN OOD AUROC from 89.74 to 91.96 and reduces corrupted-data ECE from 0.314 to 0.200.
- With ResNet-50 DEs on ImageNet vs ImageNet-O, AUROC improves from 50.20 to 54.87 (TU metric) and EU-based AUROC on CIFAR100 vs SVHN rises from 73.83 to 80.22.
- With ViT-B DEs, CIFAR10 vs SVHN AUROC increases from 77.71 to 80.71 using EU.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply the credal wrapper instead of classical BNNR averaging | BNNR, VGG16 · CIFAR10-C | ECE: 0.470 ±0.050 -> 0.416 ±0.050 | Helped |
| Apply the credal wrapper instead of classical DE averaging | DE, VGG16 · CIFAR10 vs SVHN | AUROC: 89.74 ±1.31 -> 91.96 ±1.33 | Helped |
| Apply the credal wrapper instead of classical DE averaging | DE, ResNet-50 · CIFAR100 vs SVHN | AUROC: 73.83± 1.97 -> 80.22± 1.96 | Helped |
| Apply the credal wrapper instead of classical DE averaging | DE, ResNet-50 · ImageNet vs ImageNet-O | AUROC: 65.70± 0.41 -> 66.20± 0.38 | Helped |
| Increase PIA J from 10 to 100 (ablation) | Credal wrapper, DE ResNet-50 · CIFAR100 vs SVHN | AUROC: 78.81± 1.87 -> 80.55± 1.99 | Helped |
| Increase PIA J from 10 to 100 (ablation) | Credal wrapper, DE ResNet-50 · CIFAR100 vs SVHN | time cost per sample: 7.182± 0.087 -> 109.469± 1.330 | Hurt |
| Increase DE ensemble size from N=3 to N=25 (ablation) | Credal wrapper, DE ResNet-50 · CIFAR10 vs SVHN | AUROC: 91.97± 1.38 -> 95.84± 0.15 | Helped |
| Use the credal wrapper instead of classical averaging (ablation) | DE, ResNet-50, N=25 · CIFAR10 vs SVHN | AUROC: 91.31± 0.27 -> 95.84± 0.15 | Helped |
| Use the credal wrapper instead of classical BNNR averaging (ablation) | BNNR, VGG16, N=10 · CIFAR10 vs SVHN | AUROC: 86.60 ±1.28 -> 88.38 ±1.53 | Helped |
| Use the credal wrapper instead of classical BNNR averaging (ablation) | BNNR, ResNet-18, N=1000 · CIFAR10 vs SVHN | AUROC: 87.24 ±1.72 -> 89.36 ±1.25 | Helped |
| Use generalized Hartley measure for credal-set EU (ablation) | BNNR, VGG16 · CIFAR10 vs SVHN | AUROC: 86.65± 1.26 -> 87.24± 1.32 | Helped |
Surprising findings
- Averaged DE predictions with a larger ensemble are outperformed by the credal wrapper with only 5 members, which the authors argue shows the wrapper beats simply adding more samples.
- The EDD baseline with a fair (basic) learning rate scheduler performs poorly, well below the BNN and DE baselines.
Limitations
- Uncertainty quantification requires higher computational cost than classical averaging, especially with many classes (time per sample rises with J).
- BNNs with sampling were not evaluated on large-scale datasets due to memory limits, so the large-scale experiments only use DEs.
- The authors state the method is not the best choice when memory or compute is strictly limited.
Experimental setup
Datasets: CIFAR10 CIFAR100 ImageNet
Benchmarks: CIFAR10 vs SVHN CIFAR10 vs Tiny-ImageNet CIFAR10 vs CIFAR10-C CIFAR100 vs SVHN CIFAR100 vs Tiny-ImageNet CIFAR100 vs CIFAR100-C ImageNet vs ImageNet-O
Metrics: AUROC AUPRC ECE ACC
Baselines: BNNR BNNF Deep Ensembles (averaged probability) EDD EDD-Fair
Builds on: Bayesian model averaging deep ensembles generalized Hartley measure Ensemble Distribution Distillation
Closest papers by meaning
- Credal Deep Ensembles for Uncertainty Quantification (NeurIPS 2024 · similarity 0.91)
- Conformalized Credal Set Predictors (NeurIPS 2024 · similarity 0.91)
- Credal Prediction based on Relative Likelihood (NeurIPS 2025 · similarity 0.90)
- Credal Learning Theory (NeurIPS 2024 · similarity 0.87)
- Distributional regression: CRPS-error bounds for model fitting, model selection and convex aggregation (NeurIPS 2024 · similarity 0.82)
- Estimating Epistemic and Aleatoric Uncertainty with a Single Model (NeurIPS 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.