Research map / Learning theory and algorithms / Fairness
fairret: a Framework for Differentiable Fairness Regularization Terms
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces fairret, a framework of differentiable fairness regularization terms built on linear-fractional statistics, with violation (SmoothMax) and projection (KL, JS, SED) variants usable as losses in PyTorch training.
Problem
Existing fairness toolkits support only a limited range of fairness definitions and integrate poorly with automatic differentiation, so fairness cannot easily be added as a modular loss term during training.
Approach
Quantify unfairness for many parity notions expressed as linear-fractional statistics via modular regularization terms: violation-based terms (Norm, SmoothMax) penalize constraint violation, and projection-based terms penalize divergence from a fair projection with the overall statistic fixed (c-fixed), which yields linear constraints and avoids a meta-algorithm.
Key results
- For linear statistics (DP, EO), the KL-projection fairret reduces fairness violation with minimal AUROC loss and clearly outperforms the SmoothMax violation fairret.
- For linear-fractional statistics (PP, TE), SmoothMax never makes violation worse, while the KL-projection often worsens both violation and AUROC on the test set.
- The FFB baselines (PRemover, HSIC, AdvDebias) perform worse than fairret methods on DP and EO, since they only support a single categorical sensitive attribute.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add a SmoothMax fairness regularizer | Neural classifier · DP and EO on Bank, CreditCard, LawSchool, and ACSIncome | fairness violation | Helped |
| Add a SmoothMax fairness regularizer | Neural classifier · DP and EO on Bank, CreditCard, LawSchool, and ACSIncome | AUROC | Mixed |
| Use a D_KL-projection fairret instead of SmoothMax | Neural classifier · DP and EO on Bank, CreditCard, LawSchool, and ACSIncome | fairness violation | Helped |
| Use a D_KL-projection fairret instead of SmoothMax | Neural classifier · DP and EO on Bank, CreditCard, LawSchool, and ACSIncome | AUROC | No clear effect |
| Use FFB baselines instead of fairret methods | PRemover, HSIC, and AdvDebias · DP and EO on Bank, CreditCard, LawSchool, and ACSIncome | fairness violation | Hurt |
| Use SmoothMax for PP and TE | Neural classifier · PP and TE on Bank, CreditCard, LawSchool, and ACSIncome | fairness violation | Helped |
| Use a D_KL-projection fairret for PP and TE | Neural classifier · PP and TE on Bank, CreditCard, LawSchool, and ACSIncome | fairness violation | Hurt |
| Use a D_KL-projection fairret for PP and TE | Neural classifier · PP and TE on Bank, CreditCard, LawSchool, and ACSIncome | AUROC | Hurt |
| Limit projection optimization to at most 10 iterations instead of using more than 10 (ablation) | D_KL-projection fairret · ACSIncome DP | fairness violation | Helped |
Surprising findings
- Projection fairrets often make both fairness violation and AUROC worse on the test set for PP and TE, suggesting linear-fractional notions are much harder to optimize.
- Training with at most 10 iterations per projection gave slightly lower DP violation than more iterations, possibly due to a positive effect of fewer iterations.
Limitations
- Fairness notions with linear-fractional statistics (PP, TE) are hard to optimize, with test-set degradation for projection fairrets.
- Probabilistic classifiers only approximate hard-decision fairness notions; gradient quality trades off with decision hardness.
- Mini-batch estimates of violation are biased for small batch sizes, and only per-axis (not intersectional) fairness is enforced.
Experimental setup
Datasets: Bank CreditCard LawSchool ACSIncome
Benchmarks: Bank CreditCard LawSchool ACSIncome
Metrics: AUROC fairness violation (max_k v_k) for DP, EO, PP, TE
Baselines: PRemover HSIC AdvDebias Unfair (lambda=0)
Builds on: Fairlearn Fair Fairness Benchmark (FFB) cvxpy
Closest papers by meaning
- Fairness via Independence: A General Regularization Framework for Machine Learning (ICLR 2026 · similarity 0.92)
- The Fragility of Fairness: Causal Sensitivity Analysis for Fair Machine Learning (NeurIPS 2024 · similarity 0.88)
- FairNet: Dynamic Fairness Correction without Performance Loss via Contrastive Conditional LoRA (NeurIPS 2025 · similarity 0.87)
- Fairness Overfitting in Machine Learning: An Information-Theoretic Perspective (ICML 2025 · similarity 0.87)
- Understanding Fairness and Prediction Error through Subspace Decomposition and Influence Analysis (NeurIPS 2025 · similarity 0.87)
- Kernel-based Equalized Odds: A Quantification of Accuracy-Fairness Trade-off in Fair Representation Learning (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.