Research map / Label-efficient and robust learning / Recognition and long-tail
FairNet: Dynamic Fairness Correction without Performance Loss via Contrastive Conditional LoRA
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FairNet, which gates conditionally activated LoRA adapters with per-layer bias detectors and trains them with a contrastive triplet loss to reduce minority-group disparities without hurting accuracy.
Problem
Debiasing methods typically trade off accuracy for fairness, use static global corrections, underfit minority subgroups, and handle sensitive attribute labels poorly (requiring full labels or discarding partial ones).
Approach
Instance-level, detector-gated conditional LoRA modules inside the model are activated only for samples flagged as likely minority-group members, and trained with a triplet contrastive loss that aligns minority representations with same-class majority representations. The framework supports full, partial, or no sensitive attribute labels.
Key results
- On CelebA, FairNet-Full reaches ACC 95.9%, WGA 88.2%, EOD 3.8%, beating GroupDRO (WGA 87.4%, EOD 4.7%) and matching ERM accuracy (95.8%).
- On MultiNLI, FairNet-Unlabel improves WGA from 67.3% (ERM) to 73.1% while keeping ACC at 82.5%.
- Ablation: removing the detector drops CelebA ACC from 95.9% to 94.1%, while removing the contrastive loss drops WGA from 86.5% to 81.2%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FairNet-Unlabel instead of ERM without sensitive-attribute labels | FairNet · CelebA | WGA: 77.9 -> 82.3 | Helped |
| Use FairNet-Unlabel instead of ERM without sensitive-attribute labels | FairNet · MultiNLI | WGA: 67.3 -> 73.1 | Helped |
| Use FairNet-Partial instead of GroupDRO with partial sensitive labels | FairNet-Partial · CelebA | ACC: 94.0 -> 95.9 | Helped |
| Use FairNet-Full instead of GroupDRO with full sensitive labels | FairNet-Full · CelebA | WGA: 87.4 -> 88.2 | Helped |
| Use FairNet-Full instead of GroupDRO with full sensitive labels | FairNet-Full · MultiNLI | EOD: 5.5 -> 4.7 | Helped |
| Apply the racial correction | FairNet-Partial · HateXplain, DistilBERT-base, Race | EOD: 26.5 ± 0.7 -> 24.4 ± 0.6 | Helped |
| Add gender correction after racial correction | FairNet-Partial · HateXplain, DistilBERT-base, Gender | EOD: 11.3 ± 1.1 -> 7.4 ± 0.6 | Helped |
| Add gender correction after racial correction | FairNet-Partial · HateXplain, BERT-base, Gender | EOD: 12.9 ± 1.1 -> 8.2 ± 0.4 | Helped |
| Remove the bias detector, making LoRA active on every input (ablation) | FairNet-Partial · CelebA and MultiNLI | ACC: 95.9% to 94.1%; 82.6% to 81.0% | Hurt |
| Remove the bias detector, making LoRA active on every input (ablation) | FairNet-Partial · CelebA and MultiNLI | WGA: 86.5 -> 86.7; 76.5 -> 77.2 | Mixed |
| Replace contrastive loss with binary cross entropy on flagged instances (ablation) | FairNet-Partial · CelebA and MultiNLI | WGA: 86.5% to 81.2%; 76.5% to 70.1% | Hurt |
| Remove both the detector and contrastive loss (ablation) | FairNet-Partial · CelebA and MultiNLI | WGA: 86.5 -> 82.3; 76.5 -> 71.8 | Hurt |
Surprising findings
- Removing the bias detector slightly improves WGA and EOD while lowering accuracy, indicating a fairness-accuracy tension that the gating is meant to address.
- Some reported ACC gains in HateXplain are within a few tenths of a point and differences are small relative to std.
Limitations
- Theoretical guarantee depends on assumed conditions (ratio TPR/FPR and LoRA improving minority group) that the paper argues are 'likely' rather than proven.
- Experiments limited to three datasets with small base models (8-layer ViT, DistilBERT/BERT-base).
Experimental setup
Datasets: CelebA MultiNLI HateXplain
Benchmarks: CelebA MultiNLI HateXplain
Metrics: ACC WGA EOD DP EOp
Baselines: ERM GroupDRO DFR Sebra Lu et al. D3M Eq.Odds GSTAR
Builds on: LoRA GroupDRO JTT DFR
Closest papers by meaning
- Fair Bilevel Neural Network (FairBiNN): On Balancing fairness and accuracy via Stackelberg Equilibrium (NeurIPS 2024 · similarity 0.88)
- FairICP: Encouraging Equalized Odds via Inverse Conditional Permutation (ICML 2025 · similarity 0.88)
- FairSIN: Achieving Fairness in Graph Neural Networks through Sensitive Information Neutralization (AAAI 2024 · similarity 0.88)
- The Fragility of Fairness: Causal Sensitivity Analysis for Fair Machine Learning (NeurIPS 2024 · similarity 0.88)
- fairret: a Framework for Differentiable Fairness Regularization Terms (ICLR 2024 · similarity 0.87)
- ABCFair: an Adaptable Benchmark approach for Comparing Fairness Methods (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.