Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Treeffuser: probabilistic prediction via conditional diffusions with gradient-boosted trees
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Treeffuser, a conditional diffusion model whose score function is estimated by gradient-boosted trees (LightGBM), for probabilistic prediction on tabular data.
Problem
Probabilistic prediction on tabular data: most methods assume parametric response distributions, which fail under multimodal, skewed, inflated, or multivariate responses and yield poorly calibrated uncertainty.
Approach
Learn a conditional variance-exploding diffusion whose conditional score is fit with gradient-boosted trees instead of neural networks, using a noise-scaling reparametrization (S = U/sigma) so each output dimension becomes a scalar LightGBM regression; samples are generated by Euler-Maruyama reverse SDE solving.
Key results
- On 10 UCI datasets, Treeffuser attains the lowest CRPS on most datasets (e.g., naval 2.02 vs 3.20 for iBUG; wine 2.59 vs 3.24 for quantile regression), even without tuning.
- On Walmart sales, Treeffuser achieves CRPS 6.44 vs 6.86 for NGBoost Poisson and 7.05 for deep ensembles, with comparable MAE and RMSE.
- Training on 112,000 Walmart points takes 53.2 seconds on a laptop and sampling 10,000 points takes 2.53 seconds.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Treeffuser instead of the strongest reported baseline | Probabilistic prediction on UCI datasets · UCI datasets | CRPS | Helped |
| Use Treeffuser instead of NGBoost Poisson | Walmart sales forecasting · Walmart, last 30 days | CRPS: 6.86 -> 6.44 | Helped |
| Use Treeffuser instead of Deep ensembles | Walmart sales forecasting · Walmart, last 30 days | RMSE: 2.03 -> 2.09 | Hurt |
| Use Treeffuser instead of Deep ensembles | Walmart sales forecasting · Walmart, last 30 days | MAE: 0.97 -> 0.99 | Hurt |
| Add noise-scaling reparametrization of the score function (ablation) | Treeffuser · Probabilistic prediction on real-world datasets | accuracy and stability | Helped |
Surprising findings
- CARD, a neural diffusion baseline for tabular probabilistic prediction, returned very poor results and was excluded from experiments.
- On Gaussian linear data, Treeffuser is not best; the Gaussian-likelihood deep ensemble oracle and NGBoost match or beat it, as expected.
Limitations
- Diffusion is designed for continuous responses; count data are handled imperfectly.
- No closed-form density; sampling requires solving an SDE, which is costly when many samples are needed.
- Sampling cost grows with many samples per datapoint.
Experimental setup
Datasets: UCI regression datasets (bike, energy, kin8nm, movies, naval, news, power, superconductivity, wine, yacht) Walmart M5 sales data synthetic multimodal, inflated, and multivariate distributions
Benchmarks: UCI regression datasets Walmart M5 sales synthetic distributions
Metrics: CRPS MAE RMSE posterior predictive checks newsvendor cumulative profit
Baselines: NGBoost IBUG Quantile regression Deep ensembles DRF NGBoost Poisson
Builds on: score-based diffusion models LightGBM NGBoost IBUG
Closest papers by meaning
- Partition Tree: Conditional Density Estimation over General Outcome Spaces (NeurIPS 2026 · similarity 0.86)
- TFG: Unified Training-Free Guidance for Diffusion Models (NeurIPS 2024 · similarity 0.84)
- BARK: A Fully Bayesian Tree Kernel for Black-box Optimization (ICML 2025 · similarity 0.84)
- Bayesian Nonparametrics Meets Data-Driven Distributionally Robust Optimization (NeurIPS 2024 · similarity 0.83)
- Bayesian Additive Distribution Regression (NeurIPS 2026 · similarity 0.83)
- No Training, No Problem: Rethinking Classifier-Free Guidance for Diffusion Models (ICLR 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.