Research map / Probabilistic and scientific modeling / Variational and generative sampling
Partition Tree: Conditional Density Estimation over General Outcome Spaces
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Partition Tree, a greedy log-loss tree that estimates conditional densities as piecewise-constant histograms over mixed continuous/categorical outcome spaces, plus a bagged Partition Forest, with L1 consistency results.
Problem
Conditional density estimation over general (continuous, categorical, mixed) outcome spaces with tree-based methods that are scalable and avoid parametric leaf assumptions.
Approach
Model conditional densities as piecewise-constant Radon-Nikodym derivatives on data-adaptive joint covariate-outcome partitions, grown greedily by best-first maximization of empirical conditional log-likelihood gain; ensemble by averaging densities across bagged trees.
Key results
- Partition Tree beats CART on 8 of 9 classification datasets in log-loss.
- Partition Forest beats Random Forest on most classification datasets and 8 of 10 regression datasets.
- Partition Tree outperforms CADET on Air, Power, California, and Protein; CDTree achieves the best average rank among single-tree methods.
Surprising findings
- CDTree remains more robust to correlated feature duplication than Partition Tree, which degrades gradually.
Limitations
- Partition Tree is less robust than CDTree under feature duplication in expectation.
- Pruning strategies and boosting integration left for future work.
- Tuning budget is limited (200 Optuna trials, 20-minute cap); CDTree not tuned.
Experimental setup
Datasets: Concrete Compressive Strength
Benchmarks: classification benchmarks (9 datasets) regression benchmarks (10 datasets incl. Air, Power, California, Protein) Concrete Compressive Strength
Metrics: log-loss negative log-likelihood RMSE average rank
Baselines: CART Random Forest CADET CDTree
Builds on: CART Random Forests CADET CDTree
Closest papers by meaning
- Conditional Density Estimation with Histogram Trees (NeurIPS 2024 · similarity 0.88)
- Treeffuser: probabilistic prediction via conditional diffusions with gradient-boosted trees (NeurIPS 2024 · similarity 0.86)
- Multistage Defer Trees for Hybrid Interpretability: If at First You Can't Succeed, Tree Again (NeurIPS 2026 · similarity 0.85)
- Generalized Random Forests Using Fixed-Point Trees (ICML 2025 · similarity 0.85)
- Bayesian Additive Distribution Regression (NeurIPS 2026 · similarity 0.85)
- A Likelihood Based Approach to Distribution Regression Using Conditional Deep Generative Models (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.