Research map / Learning theory and algorithms / Prediction and decision losses
Generative Forests
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces generative forests (GeoT), sets of trees whose leaves jointly define a generative density, trained by a top-down boosting algorithm that reduces to supervised two-class decision tree induction with uniform-measure negatives.
Problem
Generative modeling of tabular data: existing tree-based generators either rely on a single tree or on sampling from one tree of a forest, which is combinatorially inefficient and has restrictive theory; training guarantees for such generators are weak.
Approach
Generative forests generate each observation using leaves from all trees jointly (intersection of leaf supports), and train via a top-down greedy boosting algorithm (TopDownGT) that reduces to supervised class-probability estimation between data and uniform samples, with weak-learning-style convergence guarantees on the likelihood ratio risk for strictly proper, differentiable losses.
Key results
- On LIFELIKE with T=500 trees and J=2000 splits, GeoT beats ARF on most of the 8 domains (e.g., grid coverage 0.975 vs 0.908, density 0.995 vs 0.630) and shows significant wins over VCAE on all metrics.
- Imputation: GeoT with small models (stumps, J=T=20) can compete with MICE.RF while using about a hundred times fewer trees.
- DENSITY: GeoT is statistically better than KDE on 9 domains, worse on 3, and no significant difference on 5.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Generative Forest instead of ARF | tabular data generation · lifelike: ring | Sinkhorn: 0.285±0.008 vs 0.286±0.007 | No clear effect |
| Use Generative Forest instead of ARF | tabular data generation · lifelike: grid | Coverage: 0.975±0.009 vs 0.908±0.010 | Helped |
| Use Generative Forest instead of ARF | tabular data generation · lifelike: wred | Density: 0.961±0.026 vs 0.801±0.028 | Helped |
| Use Generative Forest instead of ARF | tabular data generation · lifelike: ring | F1 measure: 0.070±0.030 vs 0.086±0.047 | No clear effect |
| Use Generative Forest instead of CT | tabular data generation · lifelike: ring | Coverage: 0.968±0.010 vs 0.798±0.050 | Helped |
| Use Generative Forest instead of Forest Flows | tabular data generation · lifelike: grid | Sinkhorn: 0.390±0.002 vs 0.393±0.001 | Helped |
| Use Generative Forest instead of Forest Flows | tabular data generation · lifelike: ring | Sinkhorn: 0.285±0.008 vs 0.276±0.001 | No clear effect |
| Use Generative Forest instead of VCAE-G | tabular data generation · lifelike: ring | Coverage: 0.968±0.010 vs 0.364±0.037 | Helped |
| Use Generative Forest instead of KDE | density estimation · 21 experimental domains | density: 9 vs 3 domains | Mixed |
| Use Generative Forest for missing-data imputation instead of MICE-RF | missing-data imputation · tabular imputation experiments | perr / rmse | Mixed |
Surprising findings
- Very small models (e.g., sets of stumps) can match or beat much larger contenders such as ARF and Forest Flows on several domains.
- Overfitting risk when J and/or T are too large, motivating future pruning.
Limitations
- No principled way to choose the model size (T, J); performance is domain-dependent and needs cross-validation.
- Overfitting when J or T is large; no pruning currently.
- Assumes bounded domain; generalization properties not investigated.
Experimental setup
Datasets: UCI abalone OpenML analcatdata_supreme OpenML kc1 21 tabular domains (UCI, Kaggle, OpenML, Stanford Open Policing, simulated)
Benchmarks: LIFELIKE generation on 8 numerical/binary domains IMPUTE on abalone and analcatdata_supreme DENSITY vs KDE on 17 domains
Metrics: Sinkhorn coverage density F1 measure perr rmse density statistical test
Baselines: Adversarial Random Forests CT-GAN Forest Flows Vine copulas autoencoders MICE with CART and random forests kernel density estimation generative trees
Builds on: generative trees adversarial random forests copycat GAN training boosting for decision trees
Closest papers by meaning
- NRGBoost: Energy-Based Generative Boosted Trees (ICLR 2025 · similarity 0.89)
- Generative Model for Decision Trees (AAAI 2024 · similarity 0.88)
- TabSDS: a Lightweight, Fully Non-Parametric, and Model Free Approach for Generating Synthetic Tabular Data (ICML 2025 · similarity 0.87)
- GRANDE: Gradient-Based Decision Tree Ensembles for Tabular Data (ICLR 2024 · similarity 0.86)
- Tabular Foundation Model for Generative Modelling (NeurIPS 2026 · similarity 0.85)
- How Realistic Is Your Synthetic Data? Constraining Deep Generative Models for Tabular Data (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.