Research map / Architectures and efficiency / Pruning and efficient tuning
FairTune: Optimizing Parameter Efficient Fine Tuning for Fairness in Medical Image Analysis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FairTune, a bi-level optimisation framework that uses TPE with successive halving to search over PEFT parameter masks (and learning rate) of an ImageNet-pretrained ViT-B, selecting by validation worst-subgroup AUC to improve fairness on medical imaging.
Problem
Deep models fit training data perfectly, so fairness constraints on training data are ineffective; bias emerges through differential generalisation to unseen data in medical image classification. The question is which parameters to fine-tune to maximise validation fairness.
Approach
Treat the choice of PEFT update mask (which ViT modules to fine-tune) and learning rate as hyperparameters optimised in a bi-level loop against validation worst-subgroup AUC, rather than prescribing a fixed PEFT method, thereby controlling capacity for fairness.
Key results
- FairTune achieves the best average overall AUC (85.7) and min-group AUC (83.1) across datasets and attributes, versus 79.9/76.7 for full fine-tuning.
- FairTune obtains average gap 3.6 vs 4.6 for full fine-tuning, and best average overall rank (1.0).
- Ablation: 36-bit full search space with min-group objective reaches min AUC 82.2 and gap 3.4, outperforming 12-bit attention (79.5) and layer-norm (78.0) subspaces.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Select the PEFT mask using validation Min Group AUC rather than Overall AUC (ablation) | FairTune 36-bit · Fitzpatrick17K, HAM10000, Papila, OL3I, OASIS-1 (average) | Min AUC: 80.4 -> 82.2 | Helped |
| Select the PEFT mask using validation Min Group AUC rather than Overall AUC (ablation) | FairTune 36-bit · Fitzpatrick17K, HAM10000, Papila, OL3I, OASIS-1 (average) | Gap: 5.0 -> 3.4 | Helped |
| Expand the mask search from 12-bit to the full 36-bit space (ablation) | FairTune · Fitzpatrick17K, HAM10000, Papila, OL3I, OASIS-1 (average) | Min AUC: 79.5 -> 82.2; 78.0 -> 82.2 | Helped |
| Use FairTune instead of full fine-tuning | ImageNet-pretrained ViT-Base · Medical imaging benchmarks (average) | Min. AUC: 76.7 -> 83.1 | Helped |
| Use FairTune instead of full fine-tuning | ImageNet-pretrained ViT-Base · Medical imaging benchmarks (average) | Gap AUC: 4.6 -> 3.6 | Helped |
| Tune attention matrices only instead of full fine-tuning | ImageNet-pretrained ViT-Base · Medical imaging benchmarks (average) | Min. AUC: 76.7 -> 77.9 | Helped |
Surprising findings
- Training from scratch yields the second-smallest AUC gap but very poor overall performance, an instance of levelling down.
- Previously proposed fairness interventions (FairPrune, FSCL) underperform standard fine-tuning overall, consistent with prior benchmarking claims.
Limitations
- Search cost is roughly 48 GPU-hours per dataset, far above a single training run.
- Optimising min-group AUC is one metric choice; results on other fairness metrics are mixed and the 36-bit search space is hard to scale to larger masks.
- Relies on binarised sensitive attributes and labels; overlapping-attribute results are limited to CheXpert.
Experimental setup
Datasets: Fitzpatrick17K HAM10000 Papila OL3I OASIS-1 Harvard-GF3300 CheXpert
Benchmarks: Fitzpatrick17K HAM10000 Papila OL3I OASIS-1 Harvard-GF3300 CheXpert
Metrics: AUROC overall min subgroup AUROC AUROC gap equalized odds difference demographic parity difference
Baselines: Train from scratch Full fine-tuning (ERM) Linear readout Attention Tuning LayerNorm Tuning FairPrune FSCL
Builds on: Attention Tuning LayerNorm Tuning (BitFit-style PEFT) Tree-structured Parzen Estimator successive halving GroupDRO-style worst-group objective
Closest papers by meaning
- The Boundaries of Fair AI in Medical Image Prognosis: A Causal Perspective (NeurIPS 2025 · similarity 0.90)
- FairMedFM: Fairness Benchmarking for Medical Imaging Foundation Models (NeurIPS 2024 · similarity 0.89)
- FairLLaVA: Fairness-Aware Parameter-Efficient Fine-Tuning for Large Vision-Language Assistants (CVPR 2026 · similarity 0.88)
- FairCLIP: Harnessing Fairness in Vision-Language Learning (CVPR 2024 · similarity 0.88)
- FairSeg: A Large-Scale Medical Image Segmentation Dataset for Fairness Learning Using Segment Anything Model with Fair Error-Bound Scaling (ICLR 2024 · similarity 0.88)
- FairNet: Dynamic Fairness Correction without Performance Loss via Contrastive Conditional LoRA (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.