Research map / Learning theory and algorithms / Kernels and regression theory
An Investigation of Why Overparameterization Exacerbates Spurious Correlations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase model size while training with reweighting | ResNet10 on CelebA · CelebA test | worst-group error: 25.6% -> >60% | Hurt |
| Increase random-feature model size while training with reweighting | Logistic regression on Waterbirds · Waterbirds test | worst-group error: 26.6% -> 42.4% | Hurt |
| Increase model size while training with reweighting | Random features on synthetic data · Synthetic test | worst-group error: 28.5% -> around 55% | Hurt |
| Increase model size | Reweighted models on CelebA, Waterbirds, and synthetic data · CelebA, Waterbirds, synthetic | average test error | Helped |
| Train on a group-balanced subset by subsampling groups | Overparameterized ResNet10 on CelebA · CelebA test | worst-group error: 25.6% -> 15.1% | Helped |
| Use ERM instead of reweighting on the full training data (ablation) | Models across CelebA, Waterbirds, and synthetic data · CelebA, Waterbirds, synthetic | worst-group test error: >50% for CelebA; 44% for Waterbirds; >48% for synthetic | Hurt |
| Remove spurious signal by setting its feature mean to zero (ablation) | Overparameterized random-feature model on synthetic data · Synthetic test | worst-group error | Helped |
| Increase the majority fraction (ablation) | Reweighted random-feature models on synthetic data · Synthetic test | worst-group error | Hurt |
| Balance the groups by setting the majority fraction to 0.5 (ablation) | Reweighted random-feature models on synthetic data · Synthetic test | worst-group error | Helped |
| Increase the spurious-core information ratio (ablation) | Reweighted random-feature models on synthetic data · Synthetic test | worst-group error | Hurt |
| Increase L2 regularization strength (ablation) | Overparameterized reweighted models · CelebA, Waterbirds, synthetic | worst-group error | Helped |
| Increase L2 regularization strength (ablation) | Overparameterized reweighted models · Waterbirds and synthetic | average test error | Hurt |
Closest papers by meaning
- Spurious Correlations in High Dimensional Regression: The Roles of Regularization, Simplicity Bias and Over-Parameterization (ICML 2025 · similarity 0.86)
- An Effective Theory of Bias Amplification (ICLR 2025 · similarity 0.85)
- Mitigating Spurious Correlations via Disagreement Probability (NeurIPS 2024 · similarity 0.85)
- Evaluating Model Bias Requires Characterizing its Mistakes (ICML 2024 · similarity 0.85)
- The Impact of Coreset Selection on Spurious Correlations and Group Robustness (NeurIPS 2025 · similarity 0.84)
- Prediction Risk and Estimation Risk of the Ridgeless Least Squares Estimator under General Assumptions on Regression Errors (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.