Research map / Learning theory and algorithms / Prediction and decision losses
Smaller, more accurate regression forests using tree alternating optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use linear-predictor leaves instead of constant leaves in TAO trees | TAO regression forests · abalone, ailerons, cpuact, CT slice, YearPredictionMSD, SARCOS, MNIST | test RMSE | Helped |
| use TAO oblique-tree forests (bagged) instead of RF/boosting baselines | regression forests · CT slice and MNIST | test RMSE: TAO-l error about half of next best on CT slice and SARCOS | Helped |
| use all features at each node instead of random feature subsets (m=floor(D/3)) (ablation) | TAO regression forests (cpuact) · cpuact, T=10 depth 6 | test RMSE: 2.43 vs 2.79 for local m=floor(D/3) | Helped |
| train each tree on a bootstrap or ~90% random subsample instead of full data (ablation) | TAO regression forests (cpuact) · cpuact | test RMSE | Helped |
| randomly initialize each tree with different initial parameters rather than the same initial tree (ablation) | TAO regression forests (cpuact) · cpuact | test RMSE | Helped |
| increase TAO iterations beyond about 40-80 (ablation) | TAO regression forests (cpuact) · cpuact | test RMSE | Mixed |
| single TAO-l tree instead of a forest of constant-leaf TAO trees on MNIST full-image rotation | TAO regression · MNIST full-image rotation | test RMSE x10^-2: 9.63 single TAO-l vs 14.38 RF | Helped |
Closest papers by meaning
- A faster training algorithm for regression trees with linear leaves, and an analysis of its complexity (NeurIPS 2025 · similarity 0.91)
- Towards Better Decision Forests: Forest Alternating Optimization (CVPR 2023 · similarity 0.91)
- Minimax Rates and Spectral Distillation for Tree Ensembles (NeurIPS 2026 · similarity 0.87)
- Piecewise Constant and Linear Regression Trees: An Optimal Dynamic Programming Approach (ICML 2024 · similarity 0.87)
- Multistage Defer Trees for Hybrid Interpretability: If at First You Can't Succeed, Tree Again (NeurIPS 2026 · similarity 0.86)
- Compressing tree ensembles through Level-wise Optimization and Pruning (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.