Research map / Learning theory and algorithms / Prediction and decision losses
Uncertainty Regularized Evidential Regression
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that Evidential Regression Networks (ERN) cannot learn from samples in a "high uncertainty area" where activation-constrained evidence parameters saturate, and proposes an uncertainty regularization term L^U that restores non-zero gradients, extended to multivariate ERN.
Problem
Evidential regression networks (ERN) use activations such as SoftPlus/ReLU to keep evidential parameters non-negative, and this causes vanishing gradients in a high uncertainty area (alpha near 1), so samples there cannot be learned from.
Approach
Theoretically identify the high uncertainty area where the gradient w.r.t. the pre-activation of alpha vanishes, and add a regularizer -|y-gamma| * log(exp(alpha-1)-1) whose gradient w.r.t. the pre-activation equals -|y-gamma|, so training continues from those samples; the same idea is applied to multivariate ERN via its nu parameter.
Key results
- On depth estimation within HUA, calibration errors were 0.2261 (ERN), 0.2250 (NLL-ERN) and 0.0243 (UR-ERN).
- On UCI regression, UR-ERN achieved lower RMSE than ERN on most datasets (e.g., Protein 3.70 vs 4.64 for ERN), while being comparable to NLL-ERN.
- Outside HUA on depth estimation, calibration errors were 0.1366 (ERN), 0.1978 (NLL-ERN) and 0.0289 (UR-ERN).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add uncertainty regularization to ERN | ERN · Boston | RMSE: 2.97(0.27) -> 2.91(0.28) | Helped |
| Add uncertainty regularization to ERN | ERN · Concrete | RMSE: 5.76(0.16) -> 5.74(0.24) | Helped |
| Add uncertainty regularization to ERN | ERN · Energy | RMSE: 1.93(0.13) -> 1.88(0.09) | Helped |
| Add uncertainty regularization to ERN | ERN · Power | RMSE: 3.01(0.10) -> 3.02(0.08) | Hurt |
| Add uncertainty regularization to ERN | ERN · Yacht | RMSE: 1.38(0.51) -> 1.57(0.54) | Hurt |
| Add uncertainty regularization to ERN | ERN · Kin8nm | NLL: -1.36(0.03) -> -1.37(0.03) | Helped |
| Add uncertainty regularization to ERN | ERN · Navel | NLL: -6.24(0.08) -> -6.27(0.06) | Helped |
| Add uncertainty regularization to ERN | ERN · Power | NLL: 2.54(0.02) -> 2.55(0.02) | Hurt |
| Add uncertainty regularization to ERN | ERN · Protein | NLL: 2.42(0.03) -> 2.44(0.05) | Hurt |
| Add uncertainty regularization to ERN | ERN · Cubic regression within HUA | uncertainty estimation | Helped |
| Add uncertainty regularization to ERN | ERN · NYU Depth v2 within HUA | uncertainty calibration error: 0.2261, 0.2250, and 0.0243 | Helped |
| Add uncertainty regularization to ERN | ERN · NYU Depth v2 outside HUA | uncertainty calibration error: 0.1366, 0.1978, and 0.0289 | Helped |
Surprising findings
- NLL-ERN (without the evidence regularizer) achieved better NLL than UR-ERN on several UCI datasets such as Power and Yacht, and ERN's calibration outside HUA was better than UR-ERN's in depth estimation (0.1366 vs 0.0289 is worse for ERN, so not this).
Limitations
- Only theoretically analyzed the alpha parameter (gradients w.r.t. v and beta are said to follow similar fashion without full analysis)
- Regularization coefficient lambda_1 is sensitive: too small or too large values make the regularizer blow up and uncertainty estimation fails
- Results on UCI show mixed gains; UR-ERN does not beat NLL-ERN consistently on NLL and RMSE
Experimental setup
Datasets: Cubic regression synthetic data NYU Depth v2 ApolloScape (OOD) UCI regression benchmark Multivariate synthetic data (cos/sin)
Benchmarks: NYU Depth v2 UCI regression Cubic regression synthetic ApolloScape OOD
Metrics: RMSE NLL calibration error RMSE vs confidence level (cutoff curves) entropy for OOD detection
Baselines: ERN NLL-ERN Multivariate ERN
Builds on: Deep Evidential Regression Multivariate Deep Evidential Regression Evidential Deep Learning
Closest papers by meaning
- The Evidence Contraction Issue in Deep Evidential Regression: Discussion and Solution (AAAI 2024 · similarity 0.90)
- Are Uncertainty Quantification Capabilities of Evidential Deep Learning a Mirage? (NeurIPS 2024 · similarity 0.89)
- Is Epistemic Uncertainty Faithfully Represented by Evidential Deep Learning Methods? (ICML 2024 · similarity 0.87)
- Uncertainty Estimation by Flexible Evidential Deep Learning (NeurIPS 2025 · similarity 0.87)
- SURE: SUrvey REcipes for building reliable and robust deep networks (CVPR 2024 · similarity 0.86)
- Hyper Evidential Deep Learning to Quantify Composite Classification Uncertainty (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.