Research map / Learning theory and algorithms / Prediction and decision losses
Aggregated Hold-Out
Guillaume Maillard, Sylvain Arlot, Matthieu Lerasle
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase the number of splits V (ablation) | Agghoo · epsilon-regression | excess risk: most improvement between V = 1 and V = 5 | Improved |
| increase V beyond about 5-10 (ablation) | Agghoo · epsilon-regression (tau >= 0.5) | excess risk: V much larger seems useless | No clear effect |
| increase training fraction tau from 0.1 to 0.5 (ablation) | Agghoo · epsilon-regression | excess risk: 0.1 to 0.5 | Improved |
| increase tau above 0.8 (ablation) | Agghoo · epsilon-regression | excess risk: tau > 0.8 | Hurt |
| aggregate several hold-out splits instead of single hold-out | Agghoo · epsilon-regression | excess risk | Improved |
| replace CV by Agghoo with V >= 5 and tau >= 0.5 | Agghoo · epsilon-regression | excess risk: V >= 5 | Improved |
| replace CV by Agghoo with V = 1 | Agghoo · epsilon-regression | excess risk: V = 1 | Hurt |
| use aggregation instead of selection | Agghoo · epsilon-regression (SVR) | excess risk vs oracle | Improved |
| aggregate by majority vote instead of selection | Majhoo · k-NN classification | excess risk vs oracle: at least 4 times larger than oracle 0.0034 | No clear effect |
| replace CV by Majhoo with V >= 10 | Majhoo · k-NN classification | excess risk: V >= 10 | Improved |
| replace CV by Majhoo with V = 2 | Majhoo · k-NN classification | excess risk: V = 2 | Hurt |
| increase tau above 0.6 (ablation) | Majhoo · k-NN classification | excess risk: tau > 0.6 | Hurt |
Closest papers by meaning
- An Exponential Tail Bound for the Deleted Estimate (AAAI 2019 · similarity 0.87)
- Generalization Bounds with Minimal Dependency on Hypothesis Class via Distributionally Robust Optimization (NeurIPS 2022 · similarity 0.86)
- On the Efficiency of ERM in Feature Learning (NeurIPS 2024 · similarity 0.85)
- Algorithmic Stability Unleashed: Generalization Bounds with Unbounded Losses (ICML 2024 · similarity 0.85)
- Stability and Sharper Risk Bounds with Convergence Rate $\tilde{O}(1/n^2)$ (NeurIPS 2025 · similarity 0.85)
- The optimality of kernel classifiers in Sobolev space (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
