Research map / Learning theory and algorithms / Prediction and decision losses
Randomization as Regularization: A Degrees of Freedom Explanation for Random Forest Success
Lucas Mentch, Siyu Zhou
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| reduce mtry (more randomness) in random forest (ablation) | random forest · MARS and linear simulations, estimated dof | degrees of freedom | Improved |
| use random forest (mtry=0.33) instead of bagging (mtry=1) at low SNR | bagging · synthetic linear and MARS data, SNR 0.05 to 6 | test MSE difference | Improved |
| use random forest instead of bagging at high SNR | bagging · synthetic linear and MARS data, high SNR | test MSE | Hurt |
| increase SNR from low to high | optimal mtry · MARS and linear models, n=50 and n=500 | optimal mtry | Improved |
| add more noise to the response (alpha from 0 to 0.5) | random forest vs bagging · 15 real regression datasets (UCI and high-dimensional) | shifted relative test error: drops of approximately 0.2% on csm and 0.13% on pah before rising again | Improved |
| bagged forward selection instead of forward selection | forward selection · linear model Low and Medium settings, low SNR | relative test error to Bayes | Improved |
| restrict feature eligibility (mtry=0.33) in bagged forward selection (ablation) | BaggFS · linear model Low and Medium settings, low SNR | relative test error to Bayes | Improved |
| tune mtry in RandFS (ablation) | RandFS · linear model Low setting | relative test error to Bayes | Improved |
| use default RandFS (mtry=0.33) instead of lasso | lasso · linear model Low and Medium settings | relative test error to Bayes | Improved |
| use relaxed lasso instead of RandFS | RandFS · High-5 and High-10 linear settings, larger SNR | relative test error to Bayes | Hurt |
Closest papers by meaning
- Randomization Can Reduce Both Bias and Variance: A Case Study in Random Forests (JMLR 2025 · similarity 0.91)
- Exogenous Randomness Empowering Random Forests (JMLR 2026 · similarity 0.91)
- Hierarchical Shrinkage: Improving the accuracy and interpretability of tree-based models. (ICML 2022 · similarity 0.90)
- DynFrs: An Efficient Framework for Machine Unlearning in Random Forest (ICLR 2025 · similarity 0.88)
- Generalized Random Forests Using Fixed-Point Trees (ICML 2025 · similarity 0.88)
- Autoencoding Random Forests (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
