Research map / Learning theory and algorithms / Fairness
f-FERM: A Scalable Framework for Robust Fair Empirical Risk Minimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes f-FERM, a stochastic min-max (Legendre-Fenchel) framework for fair empirical risk minimization with f-divergence regularizers, extended to distributionally robust fairness under l_p and l_infinity uncertainty sets.
Problem
Fair empirical risk minimization with divergence-based regularizers is not amenable to mini-batch stochastic optimization because the regularizer's gradient is biased on batches; fairness also degrades under distribution shift between training and test data.
Approach
Rewrite f-divergence fairness regularizers via Legendre-Fenchel duality as a nonconvex-concave min-max problem linear in data points, yielding unbiased mini-batch gradients and a convergent SGDA algorithm; robustify via DRO with l_p norm balls using a Taylor/gradient-norm approximation (small shift) or a closed-form l_infinity reformulation (large shift).
Key results
- On Adult with gender and race, f-FERM maintains a strong fairness-accuracy tradeoff across batch sizes from full-batch to 2, while other methods degrade significantly at small batch sizes.
- Under label-flipping of sensitive attributes up to 20%, f-DRO and f-infinity achieve lower demographic parity violation than SOTA distribution-shift baselines.
- Under state-level shift on new Adult, f-infinity better preserves fairness and f-DRO gives better accuracy; both beat SOTA baselines.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use KL divergence with smaller λ instead of other f-divergence regularizers | f-FERM · Adult, gender and race, batch size 8 | demographic parity violation | Helped |
| Use KL divergence with smaller λ instead of other f-divergence regularizers | f-FERM · Adult, gender and race, batch size 8 | accuracy | Helped |
| Use f-FERM instead of existing benchmark methods at smaller batch sizes | f-FERM · Adult, gender and race; full-batch, batch sizes 64, 8, and 2 | fairness-accuracy tradeoff | Helped |
| Use f-DRO or f-infinity instead of state-of-the-art methods | Robust f-FERM · Adult with corrupted gender attributes, 80% test accuracy | demographic parity violation | Helped |
| Use f-DRO rather than f-infinity | Robust f-FERM · Adult with corrupted gender attributes, 80% test accuracy | demographic parity violation | Helped |
| Use f-DRO rather than f-infinity | Robust f-FERM · New Adult, evaluation across 50 states | accuracy | Helped |
| Use f-infinity rather than other approaches | Robust f-FERM · New Adult, evaluation across 50 states | fairness violation | Helped |
| Choose KL divergence rather than χ² divergence | f-DRO · Adult with corrupted gender attributes | demographic parity violation | No clear effect |
| Choose among the tested f-divergence regularizers | f-FERM · Fairness-accuracy tradeoffs across experiments | fairness-accuracy tradeoff | Mixed |
Surprising findings
- For KL divergence with small lambda, fairness and accuracy improve simultaneously on Adult with two sensitive attributes and small batch size.
Limitations
- Convergence guarantees rely on bounded probabilities and smoothness assumptions; the continuous sensitive-attribute/target case is left to future work.
- No single f-divergence uniformly dominates; choice must be tuned by cross-validation.
- The l_p robust small-shift approach relies on a Taylor approximation valid only for small distribution shifts.
Experimental setup
Datasets: Adult German COMPAS new Adult (ACS states)
Benchmarks: Adult German COMPAS new Adult
Metrics: demographic parity violation equality of opportunity violation equalized odds violation accuracy
Baselines: ERMI FERMI MMD-based fair DRO Wasserstein fair DRO IPM-based fair DRO
Builds on: ERMI fairness regularizer Rényi correlation stochastic nonconvex-concave min-max optimization sharpness-aware minimization
Closest papers by meaning
- Empirical Risk Minimization Under Fairness Constraints (NeurIPS 2018 · similarity 0.88)
- Fairness-aware Bayes Optimal Functional Classification (NeurIPS 2025 · similarity 0.87)
- Stochastic Gradient Methods for Distributionally Robust Optimization with f-divergences (NeurIPS 2016 · similarity 0.87)
- Kernel-based Equalized Odds: A Quantification of Accuracy-Fairness Trade-off in Fair Representation Learning (NeurIPS 2025 · similarity 0.86)
- SOREL: A Stochastic Algorithm for Spectral Risks Minimization (ICLR 2025 · similarity 0.85)
- Distributionally Robust Optimization with Bias and Variance Reduction (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.