Research map / Learning theory and algorithms / Prediction and decision losses
Piecewise Linear Transformation – Propagating Aleatoric Uncertainty in Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Piecewise Linear Transformation (PLT), which propagates arbitrary input probability distributions exactly through ReLU-type neural networks by exploiting their piecewise linear structure, avoiding sampling and Gaussian assumptions.
Problem
Propagating aleatoric uncertainty, represented as probability distributions over inputs, through neural networks accurately; existing sampling and function-approximation methods incur significant approximation errors and misrepresent output predictive uncertainty.
Approach
Derive an exact pushforward formula for joint piecewise PDFs over linear regions of a piecewise linear NN, approximate the input PD on a fine edgewise-simplex grid, and propagate the resulting probability mass into a histogram-like output PDF with provable error bounds that vanish as grid size shrinks.
Key results
- PLT significantly outperforms competing methods (ADF, UT, AS) in exactness across L1, L2, and Hellinger distances against a Monte Carlo ground truth.
- Theoretical results show the approximation error in the output PD converges to zero as input simplices shrink and output bins become finer.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use PLT instead of the competing propagation methods | Neural networks with aleatoric input uncertainty · Real-world classification and regression datasets | L1 distance | Helped |
| use PLT instead of the competing propagation methods | Neural networks with aleatoric input uncertainty · Real-world classification and regression datasets | L2 distance | Helped |
| use PLT instead of the competing propagation methods | Neural networks with aleatoric input uncertainty · Real-world classification and regression datasets | Hellinger distance | Helped |
| preserve dependencies between neurons rather than assume independence | Aleatoric uncertainty propagation (ADF) · Real-world classification and regression datasets | Exactness | Helped |
Surprising findings
- Assuming independence of neurons within a layer (as in ADF) leads to large approximation errors, even for Gaussian inputs.
Limitations
- Requires an already trained NN; quality of propagated PD depends on the given model.
- Applies only to NNs with piecewise linear activations; other activations must be approximated.
- Does not address uncertainty during NN training.
Experimental setup
Datasets: publicly available real-world classification and regression datasets (details in Table 2)
Benchmarks: real-world classification and regression datasets with induced aleatoric uncertainty
Metrics: L1 distance L2 distance Hellinger distance
Baselines: ADF (assumed density filtering) Unscented Transform (UT) Active Subspace (AS) Monte Carlo simulation (ground truth)
Builds on: Gast and Roth 2018 (ADF) Unscented Transform Sattelberg et al. 2020 (polytope-based NN analysis) MICE imputation
Closest papers by meaning
- Uncertainty Quantification via Stable Distribution Propagation (ICLR 2024 · similarity 0.86)
- One Step Closer to Unbiased Aleatoric Uncertainty Estimation (AAAI 2024 · similarity 0.85)
- Uncertainty Quantification for Physics-Informed Neural Networks with Extended Fiducial Inference (NeurIPS 2025 · similarity 0.84)
- Approximating Full Conformal Prediction for Neural Network Regression with Gauss-Newton Influence (ICLR 2025 · similarity 0.84)
- General Uncertainty Estimation with Delta Variances (AAAI 2025 · similarity 0.84)
- Transitional Uncertainty with Layered Intermediate Predictions (ICML 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.