Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Is Epistemic Uncertainty Faithfully Represented by Evidential Deep Learning Methods?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Theoretical and simulation analysis showing that evidential deep learning (second-order risk minimization) methods do not faithfully represent epistemic uncertainty: optimization non-identifiability, Dirac-delta convergence for outer loss, and regularizers acting as arbitrary uncertainty budgets.
Problem
Whether evidential deep learning methods, which fit second-order distributions via inner or outer expected-loss minimization, represent epistemic uncertainty quantitatively faithfully rather than only providing relative scores useful for OOD detection and similar tasks.
Approach
Defines a reference second-order distribution as the sampling distribution of the first-order empirical risk minimizer over training sets, and uses it to test EDL methods theoretically (identifiability, convergence to Dirac deltas, regularizer as an epistemic uncertainty budget) and empirically on binary classification and regression simulations.
Key results
- Outer loss minimization without regularization converges to near-Dirac distributions; alpha and beta keep growing through 15,000 epochs without convergence.
- Empirical Wasserstein-1 distance to the reference distribution remains relatively high both inside and outside training regions, and does not change much across regularization strengths lambda in [0, 0.5].
- Inner loss minimization shows large run-to-run variance in estimated alpha and beta across 40 random initializations, consistent with non-identifiability; regularization reduces this variance.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use outer loss minimization without entropy regularization | Beta second-order model · synthetic binary classification | faithfulness to reference distribution | Hurt |
| add entropy regularization to outer loss minimization (ablation) | Beta second-order model · synthetic binary classification | faithfulness to reference distribution | Mixed |
| use inner loss minimization | Beta second-order model · synthetic binary classification | faithfulness to reference distribution | Hurt |
| vary the entropy-regularization strength (ablation) | second-order models · synthetic binary classification | Wasserstein-1 distance | No clear effect |
| add entropy regularization (ablation) | outer loss minimization · synthetic binary classification, training through 15,000 epochs | convergence | Helped |
| add entropy regularization (ablation) | inner loss minimization · synthetic binary classification, 40 random initializations | variation in estimated α and β | Helped |
| use outer expectation minimization without regularization | NIG second-order model · synthetic regression | epistemic uncertainty faithfulness | Hurt |
| add negative-entropy regularization (ablation) | outer expectation minimization · synthetic regression | epistemic uncertainty faithfulness | No clear effect |
| increase training epochs without regularization (ablation) | outer loss minimization · synthetic binary classification | convergence | Hurt |
Surprising findings
- Second-order predictive distributions for Dirichlet and NIG models are non-injective, so the risk minimizer is not unique, and the epistemic uncertainty is arbitrary.
- Regularization with a fixed lambda effectively sets an arbitrary epistemic uncertainty budget, so the resulting uncertainty depends on the choice of lambda rather than the data.
Limitations
- Experiments are limited to small synthetic one-dimensional binary classification and regression setups.
- The reference distribution depends on design choices (hypothesis space, loss, optimizer) and is not a ground-truth epistemic uncertainty.
- Does not evaluate EDL on real downstream tasks such as OOD detection, where relative uncertainty may suffice.
Experimental setup
Datasets: synthetic binary classification data (x ~ Unif[0,0.5], Bernoulli theta(x)=0.5+0.4 sin(2 pi x)) synthetic regression data (x^3 + noise, sigma^2=9)
Benchmarks: synthetic binary classification simulation synthetic regression simulation
Metrics: empirical Wasserstein-1 distance to reference distribution mean prediction and confidence bounds evolution of alpha and beta over epochs
Baselines: Dirichlet-based EDL (Sensoy et al.) deep evidential regression (Amini et al.) Posterior Network Prior Networks
Builds on: evidential deep learning deep evidential regression Bayesian neural networks proper scoring rules
Closest papers by meaning
- Are Uncertainty Quantification Capabilities of Evidential Deep Learning a Mirage? (NeurIPS 2024 · similarity 0.92)
- Uncertainty Estimation by Flexible Evidential Deep Learning (NeurIPS 2025 · similarity 0.90)
- Rethinking Aleatoric and Epistemic Uncertainty (ICML 2025 · similarity 0.89)
- Epistemic Uncertainty Quantification For Pre-Trained Neural Networks (CVPR 2024 · similarity 0.88)
- Uncertainty Regularized Evidential Regression (AAAI 2024 · similarity 0.87)
- Second-Order Uncertainty Quantification: A Distance-Based Approach (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.