Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Enhancing Uncertainty Estimation and Interpretability with Bayesian Non-negative Decision Layer
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BNDL, a Bayesian non-negative factor-analysis decision layer on top of DNN feature extractors that gives uncertainty estimates and more disentangled, interpretable features, trained with a Weibull variational inference network.
Problem
Deep networks give poor uncertainty estimates and their entangled, multifaceted neurons undermine interpretability; sparse linear decision layers may hurt accuracy and lack theoretical grounding.
Approach
Replace the final softmax/dense decision layer with a conditional Bayesian non-negative factor analysis: gamma priors on factor scores and loadings, approximated by reparameterizable Weibull variational posteriors, so that sparsity and non-negativity drive disentanglement and stochastic latents drive uncertainty; partial identifiability of NMF is argued theoretically.
Key results
- ResNet-BNDL reaches 79.82% accuracy on CIFAR-100 vs 74.62% for ResNet-18 baseline, and 95.54% on CIFAR-10 vs 94.98%.
- ViT-BNDL reaches 81.29% top-1 on ImageNet-1k vs 80.33% for ViT-Base, with PAvPU 82.50.
- On ImageNet-1k, ResNet50-BNDL SEPIN@all disentanglement is 0.44 vs 0.23 for ResNet50 (scaled by 10^2).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace the ResNet-18 decision layer with BNDL | ResNet-18 · CIFAR-10 | accuracy: 94.98 $\pm$ 0.12 -> 95.54 $\pm$ 0.08 | Helped |
| Replace the ResNet-18 decision layer with BNDL | ResNet-18 · CIFAR-10 | PAvPU | Helped |
| Replace the ResNet-18 decision layer with BNDL | ResNet-18 · CIFAR-100 | accuracy: 74.62 $\pm$ 0.23 -> 79.82 $ \pm$ 0.13 | Helped |
| Replace the ResNet-18 decision layer with BNDL | ResNet-18 · CIFAR-100 | PAvPU | Helped |
| Replace the ResNet-50 decision layer with BNDL | ResNet-50 · ImageNet-1k | accuracy: 75.33 $ \pm$ 0.14 -> 77.01 $ \pm$ 0.14 | Helped |
| Replace the ResNet-50 decision layer with BNDL | ResNet-50 · ImageNet-1k | PAvPU: 76.29 $ \pm$ 0.01 -> 77.66 $ \pm$ 0.03 | Helped |
| Replace the ViT-Base decision layer with BNDL | ViT-Base · CIFAR-10 | accuracy: 95.51 $ \pm$ 0.03 -> 96.34 $ \pm$ 0.04 | Helped |
| Replace the ViT-Base decision layer with BNDL | ViT-Base · CIFAR-100 | accuracy: 84.15 $ \pm$ 0.03 -> 85.16 $ \pm$ 0.03 | Helped |
| Replace the ViT-Base decision layer with BNDL | ViT-Base · ImageNet-1k | accuracy: 80.33 -> 81.29 $ \pm$ 0.02 | Helped |
| Replace the ResNet-50 decision layer with BNDL | ResNet-50 · ImageNet-1k | SEPIN@1: 1.50 $ \pm$ 0.02 -> 2.59 $ \pm$ 0.03 | Helped |
| Apply non-negativity constraints to decision weights and input features (ablation) | ResNet-18 · CIFAR-100 | accuracy: 74.62 $ \pm$ 0.23 -> 76.73 $ \pm$ 0.16 | Helped |
| Replace the ResNet-18 decision layer with BNDL | ResNet-18 · CIFAR-100 | SEPIN@1: 3.17 $ \pm$ 0.06 -> 3.91 $ \pm$ 0.06 | Helped |
Surprising findings
- The decision layer can be made roughly two orders of magnitude sparser with minimal accuracy loss (ImageNet-1k at sparsity 0.0024 still gives 75.7% accuracy).
Limitations
- Gains in accuracy are modest and the paper states the goal is to preserve performance rather than improve it.
- Theoretical identifiability relies on assumptions (selective window, sparsity) that are argued to hold but only loosely verified.
- Evaluated only on image classification with ResNet and ViT backbones.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 ImageNet-1k Places-10
Benchmarks: CIFAR-10 CIFAR-100 ImageNet-1k
Metrics: accuracy PAvPU SEPIN@k 1-sparsity
Baselines: MC Dropout BM CARD ViT-Base ResNet Debuggable Network ResNet-NMF
Builds on: non-negative matrix factorization Debuggable Networks Bayesian last layer Weibull variational inference
Closest papers by meaning
- Adaptive Uncertainty Estimation via High-Dimensional Testing on Latent Representations (NeurIPS 2023 · similarity 0.88)
- Uncertainty Quantification for Deep Regression using Contextualised Normalizing Flows (NeurIPS 2025 · similarity 0.87)
- Make Me a BNN: A Simple Strategy for Estimating Bayesian Uncertainty from Pre-trained Models (CVPR 2024 · similarity 0.87)
- Variational Bayesian Last Layers (ICLR 2024 · similarity 0.87)
- Neural Conditional Probability for Uncertainty Quantification (NeurIPS 2024 · similarity 0.86)
- Are Uncertainty Quantification Capabilities of Evidential Deep Learning a Mirage? (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.