Research map / Learning theory and algorithms / Prediction and decision losses
Cross-Validated Variable Selection in Tree-Based Methods Improves Predictive Performance
Amichai Painsky, Saharon Rosset
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| select split variable by leave-one-out score (ALOOF) instead of training impurity | CART · Melbourne Grants (single tree) | misclassification rate: 0.245 -> 0.218 | Improved |
| use ALOOF as sub-learner | Gradient Boosting · Melbourne Grants | misclassification rate: 0.186 -> 0.165 | Improved |
| use ALOOF as sub-learner | Random Forest · Melbourne Grants | misclassification rate: 0.172 -> 0.153 | Improved |
| ALOOF instead of unlimited-K CART | CART · Auto | MSE: 9.48 -> 8.22 | Improved |
| ALOOF instead of Conditional Inference Trees | single tree · Auto | MSE: 13.24 -> 8.22 | Improved |
| ALOOF instead of unlimited-K CART | CART · Baseball catchers | MSE: 0.076 -> 0.062 | Improved |
| ALOOF instead of unlimited-K CART | CART · Boston Housing | MSE: 24.07 -> 20.83 | Improved |
| ALOOF instead of unlimited-K CART | CART · Donations | MSE: 39.30 -> 22.92 | Improved |
| use ALOOF as sub-learner | Random Forest · Donations | MSE: 21.39 -> 16.28 | Improved |
| use ALOOF as sub-learner | Gradient Boosting · Strikes | MSE: 256.31 -> 257.12 | No clear effect |
| use ALOOF as sub-learner | Random Forest · Internet years online | MSE: 0.698 -> 0.698 | No clear effect |
| limit categorical variables to K<=32 (discard high-K) | CART · Salary Posting | misclassification rate: 0.149 -> 0.330 | Hurt |
Closest papers by meaning
- Correcting Split Selection in Online Decision Trees via Anytime-Valid Inference (ICML 2026 · similarity 0.87)
- Crossing the Validation crisis: Cross-validation reduces benchmarking variance surprisingly well (NeurIPS 2026 · similarity 0.86)
- Partition Tree: Conditional Density Estimation over General Outcome Spaces (NeurIPS 2026 · similarity 0.86)
- Tree Ensemble Explainability through the Hoeffding Functional Decomposition and TreeHFD Algorithm (NeurIPS 2025 · similarity 0.85)
- Multistage Defer Trees for Hybrid Interpretability: If at First You Can't Succeed, Tree Again (NeurIPS 2026 · similarity 0.85)
- A Unified Framework for Variable Selection in Model-Based Clustering with Missing Not at Random (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
