Research map / Label-efficient and robust learning / OOD and anomaly detection
Credal Deep Ensembles for Uncertainty Quantification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Credal Deep Ensembles (CreDEs) of CreNets that output per-class lower/upper probability bounds trained with a DRO-inspired composite loss, improving accuracy, calibration, and epistemic-uncertainty-based OOD detection over deep ensembles.
Problem
Deep ensembles give single probability distributions per member and yield low-quality epistemic uncertainty estimates and lack theoretical justification; the paper targets better epistemic uncertainty quantification for classification.
Approach
CreNets replace the softmax head with an Interval SoftMax that outputs lower and upper class probabilities (credal set); training applies cross-entropy to the upper bound and a DRO-style hard-sample-selected cross-entropy to the lower bound, so interval width reflects epistemic uncertainty; ensembles average the intervals.
Key results
- On ResNet50, CreDEs-5 reaches 93.74% test accuracy on CIFAR10 vs 93.32% for DEs-5, with ECE 0.0108 vs 0.0131.
- CreDEs-5 EU-based OOD detection on CIFAR10 vs SVHN improves AUROC to 96.55% from 89.58% for DEs-5 (ResNet50).
- On ImageNet, CreDEs-5 reaches 78.51% accuracy vs 77.92% for DEs-5, and its ECE is 0.1685 vs 0.2415 for DEs-5.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace the standard deep ensemble with CreDEs using the imin prediction | ResNet50 ensemble · CIFAR10 test | accuracy (%): 93.32±0.13 -> 93.75±0.11 | Helped |
| replace the standard deep ensemble with CreDEs using the imax prediction | ResNet50 ensemble · CIFAR100 test | accuracy (%): 75.80±0.28 -> 79.65±0.19 | Helped |
| use CreDEs with imax rather than a standard deep ensemble | ResNet50 ensemble · ImageNet test | ECE: 0.2415±0.0009 -> 0.1685±0.0004 | Helped |
| use CreDEs epistemic uncertainty rather than deep-ensemble epistemic uncertainty | ResNet50 ensemble · CIFAR10 vs SVHN | AUROC: 89.58±0.93 -> 96.55±0.25 | Helped |
| use CreDEs epistemic uncertainty rather than deep-ensemble epistemic uncertainty | ResNet50 ensemble · ImageNet vs ImageNet-O | AUROC: 65.03±0.53 -> 67.82±0.06 | Helped |
| set training hyperparameter δ from 0.5 to 0.875 (ablation) | CreDEs-5 · CIFAR10 test | accuracy (%): 93.74 -> 94.57 | Helped |
| vary training hyperparameter δ across 0.5, 0.625, 0.75, 0.875, 0.9375, and 0.96875 (ablation) | CreDEs-5 · CIFAR10 vs SVHN | AUROC: 97.44 97.44 97.92 97.95 97.42 97.51 | Mixed |
| vary training hyperparameter δ across 0.5, 0.625, 0.75, 0.875, 0.9375, and 0.96875 (ablation) | CreDEs-5 · CIFAR10 vs Tiny-ImageNet | AUROC: 88.28 89.01 89.10 89.18 89.85 89.24 | Helped |
| increase PIDR hyperparameter K (ablation) | CreDEs generalized Hartley uncertainty estimation · OOD detection benchmarks | AUROC | Helped |
| use PIDR to reduce probability-interval dimension (ablation) | CreDEs generalized Hartley uncertainty estimation · OOD detection benchmarks | AUROC | Hurt |
| use PIDR to reduce probability-interval dimension (ablation) | CreDEs total-uncertainty estimation · CIFAR100 vs SVHN/Tiny-ImageNet | OOD detection performance | Hurt |
Surprising findings
- Predicting the lower-bound class (maximin) yields higher ECE on ImageNet (0.5930) than maximax, attributed to conservatism.
- Training with delta=1 collapses upper and lower bounds to single values, so the DRO sample selection is essential.
Limitations
- Neither DEs nor CreDEs suit settings with stringent memory or compute constraints.
- Credal entropy computation via numerical optimization is expensive as the number of classes grows; PIDR approximation underestimates uncertainty.
- Non-zero gap in some OOD settings (e.g., TU AUPRC drop of 0.1% on one benchmark); no statistical coverage guarantees yet.
Experimental setup
Datasets: CIFAR10 CIFAR100 ImageNet SVHN Tiny-ImageNet CIFAR10-C ImageNet-O MNIST
Benchmarks: CIFAR10 vs SVHN CIFAR10 vs Tiny-ImageNet CIFAR100 vs SVHN CIFAR100 vs Tiny-ImageNet ImageNet vs ImageNet-O CIFAR10 vs CIFAR10-C
Metrics: test accuracy expected calibration error AUROC AUPRC
Baselines: Deep Ensembles (DEs-5) DEs with DRO (DEs*-5) DEs with product-of-experts (DEsp-5) MCDropout BNN-R BNN-F
Builds on: Deep Ensembles group DRO Interval SoftMax credal inference
Closest papers by meaning
- Credal Wrapper of Model Averaging for Uncertainty Estimation in Classification (ICLR 2025 · similarity 0.91)
- Credal Ensemble Distillation for Uncertainty Quantification (AAAI 2026 · similarity 0.90)
- Credal Prediction based on Relative Likelihood (NeurIPS 2025 · similarity 0.88)
- Conformalized Credal Set Predictors (NeurIPS 2024 · similarity 0.87)
- Credal Learning Theory (NeurIPS 2024 · similarity 0.84)
- Knowledge Distillation of Uncertainty using Deep Latent Factor Model (NeurIPS 2025 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.