Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Fully Heteroscedastic Count Regression with Deep Double Poisson Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Deep Double Poisson Networks (DDPN), which output Double Poisson parameters to give fully heteroscedastic count regression, with a discrete beta-NLL variant and ensembles that outperform Poisson, NB, and Gaussian baselines.
Problem
Deep count regression lacks fully heteroscedastic models: Poisson and Negative Binomial networks restrict variance (equi- or over-dispersion), while Gaussian networks put density on infeasible continuous values for discrete targets.
Approach
Output the two parameters (mean and inverse dispersion) of a Double Poisson distribution, which gives unrestricted variance over non-negative integers, prove learnable loss attenuation for its NLL, and add a discrete beta-NLL with stop-gradient to control attenuation.
Key results
- On Length of Stay, DDPN ensemble reaches MAE 0.485 and CRPS 0.361 versus 0.571 / 0.429 for the best Faithful Gaussian ensemble baseline.
- OOD detection on Amazon Reviews: beta0.5-DDPN gets AUROC 0.887 and FPR80 0.199, versus 0.840 AUROC for Gaussian DNN.
- DDPN recovers Poisson and NegBinom data-generating distributions even when misspecified, matching or exceeding models with the correct likelihood.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use β=0.5 loss instead of standard DDPN loss (ablation) | DDPN ensemble · Length of Stay | CRPS: 0.361 -> 0.359 | Helped |
| Use β=0.5 loss instead of standard DDPN loss (ablation) | DDPN ensemble · Inventory | MAE: 0.861 -> 0.840 | Helped |
| Use β=1.0 loss instead of standard DDPN loss (ablation) | DDPN ensemble · COCO-People | MAE: 1.024 -> 0.959 | Helped |
| Use β=1.0 loss instead of standard DDPN loss (ablation) | DDPN ensemble · Reviews | CRPS: 0.268 -> 0.257 | Helped |
| Use β=0.5 loss instead of standard DDPN loss (ablation) | DDPN ensemble · Amazon Reviews OOD | AUROC: 0.854 (0.001) -> 0.887 (0.001) | Helped |
| Use β=0.5 loss instead of standard DDPN loss (ablation) | DDPN ensemble · Amazon Reviews OOD | FPR80: 0.269 (0.002) -> 0.199 (0.001) | Helped |
| Use β=1.0 loss instead of standard DDPN loss (ablation) | DDPN ensemble · Amazon Reviews OOD | AUROC: 0.854 (0.001) -> 0.870 (0.001) | Helped |
| Use β-modified loss rather than standard DDPN loss (ablation) | DDPN · real-world count datasets | MAE | Mixed |
| Combine independently trained models as a deep ensemble | DDPN · real-world count datasets | MAE | Helped |
| Use DDPN instead of the strongest non-DDPN ensemble baseline | count regression models · real-world count datasets | MAE | Helped |
Surprising findings
- Poisson and Negative Binomial DNNs fail at mean fit because their variance restrictions force high uncertainty whenever predicted counts are high.
- Initializing gamma to high values to reduce early uncertainty hurts convergence; beta-NLL recovers the true function instead.
Limitations
- Approximate Efron moment formulas can deteriorate for very small target means with large variance.
- Only one OOD experiment on a single dataset; high-frequency counts not studied.
- Training with NLL was unstable with SGD and Adam, requiring AdamW and beta-NLL for stability.
Experimental setup
Datasets: Length of Stay COCO-People Inventory Amazon Reviews (Patio, Lawn, and Garden) King James Bible verses (OOD)
Benchmarks: Length of Stay COCO-People Inventory Amazon Reviews
Metrics: MAE CRPS AUROC AUPR FPR80
Baselines: Poisson DNN Negative Binomial DNN Gaussian DNN Faithful Gaussian Natural Gaussian beta-Gaussian
Builds on: Deep ensembles beta-NLL Double Poisson distribution Gaussian heteroscedastic regression
Closest papers by meaning
- Effective Bayesian Heteroscedastic Regression with Deep Neural Networks (NeurIPS 2023 · similarity 0.87)
- Deep Poisson gamma dynamical systems (NeurIPS 2018 · similarity 0.86)
- Deep Heteroskedastic Regression: Post-Hoc Variance Estimation from Latent Representations (NeurIPS 2026 · similarity 0.86)
- Single-Model Uncertainties for Deep Learning (NeurIPS 2019 · similarity 0.85)
- SURE: SUrvey REcipes for building reliable and robust deep networks (CVPR 2024 · similarity 0.84)
- Uncertainty Regularized Evidential Regression (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.