Research map / Label-efficient and robust learning / Recognition and long-tail
An Effective Theory of Bias Amplification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Derives a rigorous high-dimensional bias-variance theory, via operator-valued free probability, for ridge regression (with and without random projections) trained on Gaussian mixtures, explaining bias amplification and minority-group bias across parameterization regimes.
Problem
Bias amplification (a single model trained on mixed groups amplifying group disparities relative to separate models) and minority-group bias from overparameterization are not mechanistically understood, and existing theory does not characterize how model size interacts with data properties.
Approach
Precise high-dimensional test-risk characterization of ridge regression on a two-group Gaussian mixture, using operator-valued free probability theory, with random projections as a tractable proxy for model size; it defines ODD/EDD/ADD metrics and shows bias amplification and deamplification depend on feature-to-sample and parameter-to-sample ratios, regularization, and label noise.
Key results
- Theory predicts bias amplification even with balanced groups and no spurious correlations, matching empirical ODD/EDD/ADD on synthetic data.
- On Colored MNIST with disparate label noise (0.05 vs 0.25), the CNN's ODD is relatively low while EDD is much higher as training time increases, i.e., bias deamplification, as predicted by the theory.
- Optimal regularization or training time may avoid bias amplification in overparameterized models; overparameterization with extraneous features increases test-risk disparity between groups.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train one model on both groups instead of separate group models | Ridge regression with random projections · Balanced groups without spurious correlations | ADD: ADD > 1 | Mixed |
| Increase the parameter-to-sample ratio when the feature-to-sample ratio is greater than 1 | Ridge regression with random projections · Isotropic covariance | ODD: peaking at the interpolation threshold at ψ = 1 | Mixed |
| Increase the parameter-to-sample ratio when the feature-to-sample ratio is close to 1 | Ridge regression with random projections · Isotropic covariance | ADD: ADD ≫ 1 | Helped |
| Increase the parameter-to-sample ratio when the feature-to-sample ratio is greater than 1 | Ridge regression with random projections · Isotropic covariance | ADD: ADD > 1 | Helped |
| Increase the feature-to-sample ratio toward 1 in the overparameterized regime | Ridge regression with random projections · Isotropic covariance | ADD: greater than 1 | Helped |
| Increase the label-noise ratio between groups | Ridge regression with random projections · Isotropic covariance; overparameterized regime | ODD: c = σ₂² / σ₁² | Helped |
| Increase regularization or shorten training | Ridge regression with random projections · Bias amplification setup | ADD: ADD ≈ 1 | Helped |
| Decrease regularization or increase training time when overparameterized | Ridge regression with random projections · Bias amplification setup, ψ > 1 | ADD: ψ > 1; ADD > 1 | Hurt |
| Decrease regularization or increase training time when underparameterized | Ridge regression with random projections · Bias amplification setup, ψ < 1 | ADD: ψ < 1 | Helped |
| Train a CNN longer on Colored MNIST with different group label-noise rates | CNN · Colored MNIST | ODD: 0.05; 0.25 | No clear effect |
| Use one shared model rather than separate models for groups with different label-noise rates | CNN · Colored MNIST | ADD: 0.05 / 2 · V = 0.025 V; 0.25 / 2 · V = 0.125 V; ODD ≈ 0; EDD ≈ 0.1 |V| | Helped |
| Increase the feature-to-sample ratio | Single-model ridge regression with random projections · Minority-group test risk | R₂: culminating at ψ = 1 | Hurt |
Surprising findings
- Bias amplification occurs even with balanced groups and without spurious correlations.
- Overparameterization does not necessarily alleviate test-error differences between groups when extraneous features are present.
Limitations
- Assumes approximately Gaussian data and commuting (simultaneously diagonalizable) covariances and ground-truth weights.
- Linear activation random projections only approximate neural networks; theory not directly extendable to other proportionate scaling limits.
- Training time mapped to regularization via lambda = 1/t is a simplistic calibration.
Experimental setup
Datasets: Colored MNIST
Benchmarks: Colored MNIST
Metrics: observed difficulty disparity (ODD) expected difficulty disparity (EDD) amplification of difficulty disparity (ADD) test risk per group
Baselines: separate per-group models
Builds on: operator-valued free probability theory double descent random features models
Closest papers by meaning
- When majority rules, minority loses: bias amplification of gradient descent (NeurIPS 2025 · similarity 0.89)
- Mind the Gap: A Causal Perspective on Bias Amplification in Prediction & Decision-Making (NeurIPS 2024 · similarity 0.88)
- A Simple Remedy for Dataset Bias via Self-Influence: A Mislabeled Sample Perspective (NeurIPS 2024 · similarity 0.87)
- Diffusing DeBias: Synthetic Bias Amplification for Model Debiasing (NeurIPS 2025 · similarity 0.87)
- The Impact of Coreset Selection on Spurious Correlations and Group Robustness (NeurIPS 2025 · similarity 0.86)
- Evaluating Model Bias Requires Characterizing its Mistakes (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.