Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Wasserstein Gradient Boosting: A Framework for Distribution-Valued Supervised Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Wasserstein gradient boosting (WGBoost), which fits weak learners to Wasserstein gradients of KL-type loss functionals to output particle-based distributions, and applies it to tree-based evidential learning (WEvidential) for regression and classification with OOD detection.
Problem
Supervised learning where each training output is a probability distribution (e.g., individual-level posteriors of a response parameter) lacks a nonparametric, tree-based model; existing evidential learning is limited to neural networks and closed-form posteriors.
Approach
Combine gradient boosting with Wasserstein gradient flows: N particle-valued boosting ensembles are each fit to a kernel-smoothed (and diagonal second-order) Wasserstein gradient of KL divergence to each training output distribution, so the model returns particles approximating p(theta|x) without needing normalising constants.
Key results
- On the UCI regression benchmark, WEvidential achieves the best or near-best NLL on most datasets, e.g., energy NLL 0.53 ± 0.08 vs 1.39 to 2.30 for other methods except NGBoost (0.60 ± 0.45).
- On the segment dataset, WEvidential reaches OOD PR-AUC 99.67 ± 0.2 versus 96.74 ± 0.9 for Posterior Network and 35.83 to 58.13 for other baselines, while accuracy is 96.57 vs 97.27 for DEnsemble.
- On sensorless, WEvidential achieves the best accuracy (99.54 ± 0.1) but OOD PR-AUC 81.13 ± 5.3 is below Posterior Network's 88.65 ± 0.4.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use WEvidential instead of the listed uncertainty baselines | Probabilistic regression · UCI regression datasets | NLL | Mixed |
| Use WEvidential instead of the listed uncertainty baselines | Probabilistic regression · concrete, energy, power, yacht | NLL | Helped |
| Use WEvidential instead of the listed uncertainty baselines | Probabilistic regression · kin8nm, naval, protein, wine | NLL | Hurt |
| Use WEvidential instead of the listed uncertainty baselines | Probabilistic regression · concrete, energy, power, yacht | RMSE | Helped |
| Use WEvidential instead of the listed uncertainty baselines | Probabilistic regression · kin8nm, protein | RMSE | Hurt |
| Use WEvidential instead of the listed uncertainty baselines | Probabilistic regression · naval, wine | RMSE | No clear effect |
| Use WEvidential for classification | Classification · segment | Accuracy | Mixed |
| Use WEvidential for classification | Classification · sensorless | Accuracy | Helped |
| Use WEvidential particle dispersion as an OOD score | OOD detection · segment | PR-AUC | Helped |
| Use WEvidential particle dispersion as an OOD score | OOD detection · sensorless | PR-AUC | Hurt |
Surprising findings
- Posterior Network outperforms WEvidential on sensorless OOD detection, so WEvidential does not dominate across all settings.
- On the UCI benchmark, WEvidential is not uniformly best; DEvidential or CDropout win on several datasets such as kin8nm and naval RMSE.
Limitations
- Convergence properties are not established theoretically.
- Robustness to misspecified response distributions is not evaluated.
- Method is designed for tabular data; applicability to non-tabular data is unclear.
Experimental setup
Datasets: UCI regression benchmark datasets (boston, concrete, energy, kin8nm, naval, power, protein, wine, yacht) segment sensorless bone mineral density old faithful geyser
Benchmarks: UCI probabilistic regression benchmark (NLL, RMSE) segment and sensorless classification and OOD detection (accuracy, PR-AUC)
Metrics: negative log likelihood RMSE classification accuracy PR-AUC
Baselines: MCDropout Deep Ensemble Concrete Dropout NGBoost Deep Evidential Regression Distributional Distillation Posterior Network
Builds on: gradient boosting Stein variational gradient descent NGBoost deep evidential regression XGBoost
Closest papers by meaning
- Uncertainty in Gradient Boosting via Ensembles (ICLR 2021 · similarity 0.87)
- Wasserstein Distributional Normalization For Robust Distributional Certification of Noisy Labeled Data (ICML 2021 · similarity 0.87)
- Flowing Datasets with Wasserstein over Wasserstein Gradient Flows (ICML 2025 · similarity 0.85)
- SimPro: A Simple Probabilistic Framework Towards Realistic Long-Tailed Semi-Supervised Learning (ICML 2024 · similarity 0.85)
- Statistical Inference for Gradient Boosting Regression (NeurIPS 2025 · similarity 0.84)
- Universal generalization guarantees for Wasserstein distributionally robust models (ICLR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.