Research map / Learning theory and algorithms / Fairness
OxonFair: A Flexible Toolkit for Algorithmic Fairness
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces OxonFair, an open-source toolkit that enforces group fairness in binary classification via per-group thresholding tuned on validation data, supporting tabular, NLP, and vision models and arbitrary TP/FP/FN/TN-based objectives.
Problem
Enforcing group fairness in binary classification is limited in existing toolkits: they overfit when constraints are enforced on training data, support few objectives/metrics, and do not cover NLP or computer vision.
Approach
Per-group threshold post-processing with a fast cumulative-sum grid search over TP/FP/FN/TN that jointly optimizes a user-chosen objective subject to a fairness constraint, fitted on held-out validation data; for deep networks, two heads are merged into one linear layer after thresholds are chosen.
Key results
- On CelebA gender-independent/dependent/inconsistent attributes, OxonFair DEO reaches 3.92 on gender-dependent attributes versus 26.4 for ERM and 6.50 for the best prior domain-independent method, at accuracy 85.8.
- On the multilingual Twitter gender task, OxonFair (F1-optimized) reaches DEO 2.10 with F1 44.6, compared with Base F1 40.8 and DEO 21.4.
- OxonFair supports all 9 decision-based group metrics from Verma and Rubin and all 10 Sagemaker Clarify metrics, and its grid search is O(T^k + n log n) versus naive O(n T^k).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Optimize accuracy subject to DEO fairness | OxonFair on CelebA gender-independent attributes · CelebA gender-independent attributes | DEO: 16.47 -> 3.21 | Helped |
| Optimize accuracy subject to DEO fairness | OxonFair on CelebA gender-dependent attributes · CelebA gender-dependent attributes | DEO: 26.4 -> 3.92 | Helped |
| Optimize accuracy subject to DEO fairness | OxonFair on CelebA inconsistently labelled attributes · CelebA inconsistently labelled attributes | DEO: 21.9 -> 2.36 | Helped |
| Optimize F1 while enforcing validation DEO under 5% | BERT on multilingual Twitter gender · Multilingual Twitter, gender | F1: 40.8 -> 44.6 | Helped |
| Optimize balanced accuracy while enforcing validation DEO under 5% | BERT on multilingual Twitter gender · Multilingual Twitter, gender | Balanced Acc.: 63.2 -> 71.2 | Helped |
| Use OxonFair to enforce fairness | BERT on multilingual Twitter gender · Multilingual Twitter, gender | DEO: 21.4 -> 2.10 | Helped |
| Optimize accuracy subject to validation DEO under 5% | BERT on Jigsaw religion · Jigsaw, religion | Acc.: 75.0 -> 89.2 | Helped |
| Use OxonFair to enforce fairness | BERT on Jigsaw religion · Jigsaw, religion | DEO: 7.33 -> 2.36 | Helped |
| Use true rather than inferred group membership at test time (ablation) | OxonFair on multilingual Twitter and Jigsaw · NLP datasets | performance and fairness metrics | No clear effect |
| Use hard group assignment rather than soft assignment (ablation) | OxonFair with inferred characteristics · Inferred-characteristic experiments | performance | No clear effect |
| Use hard group assignment rather than soft assignment (ablation) | OxonFair with inferred characteristics · Inferred-characteristic experiments | runtime | Helped |
| Add a separate don't-know inferred group (ablation) | OxonFair fast pathway · Tabular experiments | Pareto frontier | Helped |
Surprising findings
- Jointly optimizing an objective alongside fairness can improve performance over inadequately tuned unfair baselines.
- Fairlearn-style reductions fail to enforce fairness on tabular models (XGBoost, decision trees, random forests) that reach zero training error with default parameters.
Limitations
- Exponential complexity in the number of groups limits scalability beyond a handful of groups.
- Solutions for equalized odds are known to be suboptimal under per-group thresholding.
- NLP results are weaker due to scarce positive labels and difficulty inferring group membership from short text; group labels are coarse and may not match self-identification.
Experimental setup
Datasets: Adult COMPAS CelebA Multilingual Twitter hate speech corpus Jigsaw toxicity Fitzpatrick-17k medical dataset (XGBoost experiments)
Benchmarks: Adult COMPAS CelebA Multilingual Twitter Jigsaw
Metrics: accuracy balanced accuracy F1 difference in equal opportunity (DEO) demographic parity minimum group recall min accuracy
Baselines: Fairlearn reductions AIF360 methods Fairret Cruz and Hardt LP post-processing Domain Independent Domain Discriminative Uniconf. Adversarial FairMixup CDA DP regularization EO regularization Dropout
Builds on: Fairlearn reductions AIF360 per-group thresholding Wang et al. multitask fairness head approach Zietlow et al. FairMixup benchmark setup
Closest papers by meaning
- ABCFair: an Adaptable Benchmark approach for Comparing Fairness Methods (NeurIPS 2024 · similarity 0.89)
- FairNet: Dynamic Fairness Correction without Performance Loss via Contrastive Conditional LoRA (NeurIPS 2025 · similarity 0.86)
- FFB: A Fair Fairness Benchmark for In-Processing Group Fairness Methods (ICLR 2024 · similarity 0.86)
- Post-hoc bias scoring is optimal for fair classification (ICLR 2024 · similarity 0.86)
- On the Maximal Local Disparity of Fairness-Aware Classifiers (ICML 2024 · similarity 0.85)
- Arbitrariness and Social Prediction: The Confounding Role of Variance in Fair Classification (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.