Research map / Learning theory and algorithms / Kernels and regression theory
Stability and L2-penalty in Model Averaging
Hengkun Zhu, Guohua Zou
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add L2-penalty on model weights to Mallows averaging (RM vs MM) | Mallows model averaging · wage1 Case 1, n=110 | mean test MSE: 0.169 -> 0.164 | Improved |
| add L2-penalty on model weights to jackknife averaging (RJ vs JM) | jackknife model averaging · wage1 Case 1, n=110 | mean test MSE: 0.168 -> 0.164 | Improved |
| add L2-penalty to Mallows averaging | Mallows model averaging · wage1 Case 2, n=110 | mean test MSE: 0.158 -> 0.152 | Improved |
| add L2-penalty to Mallows averaging | Mallows model averaging · wage1 Case 2, n=420 | mean test MSE: 0.146 -> 0.143 | Improved |
| add L2-penalty to Mallows averaging | Mallows model averaging · wage1 Case 1, n=420 | mean test MSE: 0.148 -> 0.147 | Improved |
| use RM instead of AIC model selection | AIC selection · wage1 Case 1, n=110 | mean test MSE: 0.185 -> 0.164 | Improved |
| use RM/RJ instead of smoothed AIC at large n | smoothed AIC · wage1 Case 2, n=420 | best performance rate: 0.370 -> 0.265 | Hurt |
| use RM instead of smoothed AIC at large n | smoothed AIC · wage1 Case 1, n=420 | best performance rate: 0.175 -> 0.215 | Hurt |
| use RJ instead of RM for best performance rate | RM · wage1 Case 2, n=110 | best performance rate: 0.400 -> 0.465 | Improved |
| increase sample size n from 110 to 420 for RM/RJ over other methods | RM and RJ · simulations, alpha=0.5, large R2 | normalized MSE | Hurt |
| use RM/RJ instead of other methods in nested simulation with R2 <= 0.5 | other averaging and selection methods · nested simulation, alpha=0.5 | normalized MSE | Improved |
Closest papers by meaning
- Model Averaging Is Asymptotically Better Than Model Selection For Prediction (JMLR 2022 · similarity 0.88)
- Instability, Computational Efficiency and Statistical Accuracy (JMLR 2025 · similarity 0.85)
- Least Squares Model Averaging for Distributed Data (JMLR 2023 · similarity 0.85)
- Model Stock: All we need is just a few fine-tuned models (ECCV 2024 · similarity 0.85)
- Generalization Analysis of Stochastic Weight Averaging with General Sampling (ICML 2024 · similarity 0.84)
- Quantifying Prediction Consistency Under Fine-tuning Multiplicity in Tabular LLMs (ICML 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
