Research map / Learning theory and algorithms / Prediction and decision losses
Getting Better from Worse: Augmented Bagging and A Cautionary Tale of Variable Importance
Lucas Mentch, Siyu Zhou
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add independent noise features to bagging (AugBagg) | Bagging with random forest comparison · Linear simulation, SNR 0.01 | relative test error: surpasses the best random forest at around q=25 | Improved |
| add independent noise features to bagging (AugBagg) | AugBagg · Linear simulation, SNR 0.14 | relative test error: helps until approximately q=50, then deteriorates | Mixed |
| make noise features correlated with an original feature (ablation) | AugBagg · Linear simulation, SNR 0.01 and 0.05 | relative test error: all AugBagg models outperform best random forest once q>50, except r=0.99 | Improved |
| add noise features correlated with original features (ablation) | AugBagg · Linear simulation, SNR 1.22 and 2.07 | relative test error: AB(q,0.99) is the only model not deteriorating substantially with q | Hurt |
| tuned AugBagg instead of bagging on original data | Bagging · 14 real datasets with injected response noise | relative test error: outperforms bagging on 12 of 14 datasets | Improved |
| add noise features on high-dimensional datasets AquaticTox and mtp2 | Bagging · AquaticTox and mtp2 | relative test error: bagging remains superior; lasso selects 17% and 4% of features | Hurt |
| drop the tested noise features instead of replacing them (ablation) | Variable importance test · Independent noise features, low SNR, large q | rejection probability: rejects nearly 30% versus nominal 5% | Hurt |
| use replacement tests with matching noise distribution (ablation) | Variable importance test · Replacement tests, same distribution | rejection probability: rejection rate stays near nominal 5% | Improved |
| replace correlated noise (r=0.7) with independent noise in replacement test (ablation) | Variable importance test · Mismatched replacement distribution | rejection probability: very high rejection probability across all but lowest SNRs, rising with q | Hurt |
Closest papers by meaning
- Randomization Can Reduce Both Bias and Variance: A Case Study in Random Forests (JMLR 2025 · similarity 0.89)
- Quantifying Uncertainty in Random Forests via Confidence Intervals and Hypothesis Tests (JMLR 2016 · similarity 0.87)
- DynFrs: An Efficient Framework for Machine Unlearning in Random Forest (ICLR 2025 · similarity 0.85)
- Data Mixture Optimization: A Multi-fidelity Multi-scale Bayesian Framework (NeurIPS 2025 · similarity 0.85)
- Generalized Random Forests Using Fixed-Point Trees (ICML 2025 · similarity 0.85)
- Importance Sampling for Nonlinear Models (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
