Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Uncertainty Quantification via Stable Distribution Propagation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Stable Distribution Propagation (SDP), which propagates Gaussian and Cauchy distributions through ReLU networks via local linearization (TV-optimal for single ReLUs), and uses pairwise distribution losses for classification.
Problem
Quantifying how input uncertainties propagate through neural networks to output uncertainties, without expensive Monte Carlo sampling or moment matching that alters pre-trained network means.
Approach
Propagate location through the network as usual and scale via local linearization (Jacobian-based scale propagation) for ReLU and other non-linearities; this is provably TV-optimal for a single ReLU on stable distributions and extends to Cauchy distributions, which moment matching cannot handle.
Key results
- On the 5-layer Iris network at sigma=0.1, SDP achieves 0.979 intersection of probabilities versus 0.972 for DVIA and 0.808 for 100-sample MC estimation.
- On UCI calibrated intervals, SDP + PNN has the narrowest intervals on 5 of 8 datasets while keeping PICP near 95%.
- On MNIST vs EMNIST selective prediction, pairwise Cauchy scores with uncertainty-aware training reach 18.3% RCAUC, the best among the compared methods.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use full-covariance SDP instead of DVIA | Gaussian propagation through a 4-ReLU-layer network · Iris, σ=10 | 1−TV: 0.703 ± 0.036 -> 0.758 ± 0.040 | Helped |
| Use full-covariance SDP instead of marginal SDP | Gaussian propagation through a 4-ReLU-layer network · Iris, σ=10 | 1−TV: 0.218 ± 0.024 -> 0.758 ± 0.040 | Helped |
| Use full-covariance SDP instead of marginal moment matching | Gaussian propagation through a 4-ReLU-layer network · Iris, σ=10 | 1−TV: 0.197 ± 0.017 -> 0.758 ± 0.040 | Helped |
| Use SDP rather than marginal moment matching | Gaussian propagation through a 4-ReLU-layer network · Iris, σ=1 | W1 Wasserstein distance: 20.9889 ± 7.7142 -> 7.7325 ± 5.1254 | Helped |
| Use SDP rather than DVIA | Gaussian propagation through a 4-ReLU-layer network · Iris, σ=1 | W1 Wasserstein distance: 7.5794 ± 4.4295 -> 7.7325 ± 5.1254 | Mixed |
| Use full-covariance SDP instead of marginal SDP | Gaussian propagation through ResNet-18 · CIFAR-10, σ=0.05 | TV: 0.02164 -> 0.01140 | Helped |
| Use full-covariance SDP instead of marginal moment matching | Gaussian propagation through ResNet-18 · CIFAR-10, σ=0.05 | TV: 0.02434 -> 0.01140 | Helped |
| Add input-uncertainty propagation to a PNN | UCI regression prediction intervals · Concrete | MPIW: 0.25 ±0.02 -> 0.28 ±0.01 | Hurt |
| Add input-uncertainty propagation to a PNN | UCI regression prediction intervals · Power | MPIW: 0.20 ±0.00 -> 0.15 ±0.01 | Helped |
| Use pairwise Cauchy prediction scores instead of pairwise Gaussian scores | Pretrained softmax model · MNIST with EMNIST letters OOD | RCAUC: 20.4% -> 19.2% | Helped |
| Train with a pairwise Gaussian distribution loss and evaluate with pairwise Cauchy scores | MNIST selective prediction model · MNIST with EMNIST letters OOD | RCAUC: 19.0% -> 18.3% | Helped |
| Add full-covariance propagation rather than marginal propagation | Distribution propagation on a 3 Layer CNN · CIFAR-10, batch size 128 | runtime per epoch: 6.31s -> 20.8s | Hurt |
Surprising findings
- Marginal moment matching beats marginal SDP at small input sigma (0.1) on the Iris setting in some cases, showing SDP is not uniformly superior.
- Propagating covariances with SDP gives large gains over marginal propagation in deep networks, where marginal methods underestimate output scale.
Limitations
- Full (non-marginal) SDP is substantially more expensive than marginal propagation, e.g., 251s vs 28.5s per epoch on ResNet-18.
- Marginal propagation discards correlations, causing layer-wise decay of output scales.
- Moment matching is not defined for GELU/SiLU and DVIA only applies to ReLU, limiting comparisons.
Experimental setup
Datasets: Iris CIFAR-10 MNIST EMNIST letters UCI regression datasets (concrete, power, wine, yacht, naval, energy, boston, kin8nm)
Benchmarks: Iris (TV / Wasserstein vs MC oracle) CIFAR-10 ResNet-18 TV UCI regression PICP/MPIW MNIST vs EMNIST selective prediction RCAUC
Metrics: intersection of probabilities (1-TV) Wasserstein-1 distance PICP MPIW risk-coverage AUC runtime per epoch
Baselines: DVIA marginal moment matching MC Estimate PNN SQR Orthonormal Certificates Dirichlet distribution outputs softmax entropy
Builds on: deterministic variational inference approximation (DVIA) moment matching probabilistic neural networks softmax cross-entropy
Closest papers by meaning
- Uncertainty Quantification with the Empirical Neural Tangent Kernel (NeurIPS 2025 · similarity 0.86)
- Piecewise Linear Transformation – Propagating Aleatoric Uncertainty in Neural Networks (AAAI 2024 · similarity 0.86)
- Neural Conditional Probability for Uncertainty Quantification (NeurIPS 2024 · similarity 0.86)
- Second-Order Uncertainty Quantification: A Distance-Based Approach (ICML 2024 · similarity 0.86)
- General Uncertainty Estimation with Delta Variances (AAAI 2025 · similarity 0.85)
- Quantifying Uncertainty in the Presence of Distribution Shifts (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.