Research map / Learning theory and algorithms / Prediction and decision losses
Understanding Why Generalized Reweighting Does Not Improve Over ERM
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use generalized reweighting instead of ERM | Overparameterized linear regression with squared loss · Training data with linearly independent inputs | Converged model | No clear effect |
| Use generalized reweighting instead of ERM | Sufficiently wide fully-connected neural network with squared loss · Unit-ball test inputs | Model output difference: O(\tilde{d}^{-1/4}) | No clear effect |
| Add small L2 regularization while retaining near-zero training risk (ablation) | Wide neural network trained with generalized reweighting · Unit-ball test inputs | Model output difference from ERM: O(\tilde{d}^{-1/4} + \sqrt{\epsilon}) | No clear effect |
| Increase L2 regularization enough to lower training performance (ablation) | Wide neural network trained with generalized reweighting · Six MNIST images | Model-weight difference | Mixed |
| Use importance weighting instead of ERM | Linear model · Six MNIST images, squared loss | L2 norm of model weights: about 0.63 | No clear effect |
| Use Group DRO instead of ERM | Linear model · Six MNIST images, squared loss | L2 norm of model weights: about 0.63 | No clear effect |
| Use importance weighting instead of ERM | Linear classifier trained with logistic loss · Separable classification | Limiting classifier direction | No clear effect |
| Use Group DRO instead of ERM | Linear classifier trained with logistic loss · Separable classification | Limiting classifier direction | No clear effect |
| Replace logistic loss with a polynomially-tailed loss (ablation) | Linear classifier trained with generalized reweighting · Six MNIST images | L2 norm of normalized model-weight gap: around 0.06 -> larger than 0.22 | Helped |
Closest papers by meaning
- On the Asymptotic Distribution of the Minimum Empirical Risk (ICML 2024 · similarity 0.87)
- The Price of Implicit Bias in Adversarially Robust Generalization (NeurIPS 2024 · similarity 0.87)
- Understanding Domain Generalization: A Noise Robustness Perspective (ICLR 2024 · similarity 0.87)
- Scaling laws for learning with real and surrogate data (NeurIPS 2024 · similarity 0.86)
- Structure-informed Risk Minimization for Robust Ensemble Learning (ICML 2025 · similarity 0.86)
- Statistical Properties of Robust Satisficing (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.