AI Paper Atlas

Research map / Learning theory and algorithms / Prediction and decision losses

Randomization as Regularization: A Degrees of Freedom Explanation for Random Forest Success

Lucas Mentch, Siyu Zhou

JMLR 2020 · Learning theory and algorithms › Prediction and decision losses · approach: Tree ensemble optimization · 56 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
reduce mtry (more randomness) in random forest (ablation)random forest · MARS and linear simulations, estimated dofdegrees of freedomImproved
use random forest (mtry=0.33) instead of bagging (mtry=1) at low SNRbagging · synthetic linear and MARS data, SNR 0.05 to 6test MSE differenceImproved
use random forest instead of bagging at high SNRbagging · synthetic linear and MARS data, high SNRtest MSEHurt
increase SNR from low to highoptimal mtry · MARS and linear models, n=50 and n=500optimal mtryImproved
add more noise to the response (alpha from 0 to 0.5)random forest vs bagging · 15 real regression datasets (UCI and high-dimensional)shifted relative test error: drops of approximately 0.2% on csm and 0.13% on pah before rising againImproved
bagged forward selection instead of forward selectionforward selection · linear model Low and Medium settings, low SNRrelative test error to BayesImproved
restrict feature eligibility (mtry=0.33) in bagged forward selection (ablation)BaggFS · linear model Low and Medium settings, low SNRrelative test error to BayesImproved
tune mtry in RandFS (ablation)RandFS · linear model Low settingrelative test error to BayesImproved
use default RandFS (mtry=0.33) instead of lassolasso · linear model Low and Medium settingsrelative test error to BayesImproved
use relaxed lasso instead of RandFSRandFS · High-5 and High-10 linear settings, larger SNRrelative test error to BayesHurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy