Research map / Label-efficient and robust learning / Recognition and long-tail

FairMedFM: Fairness Benchmarking for Medical Imaging Foundation Models

NeurIPS 2024 · Label-efficient and robust learning › Recognition and long-tail · approach: Dense features and foundation priors · 14 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces FairMedFM, a fairness benchmark and codebase evaluating 20 medical and general foundation models across 17 medical imaging datasets for classification and segmentation under LP, PEFT, CLIP zero-shot/adapt, and prompt-based segmentation, plus bias mitigation methods.

Problem

Fairness of foundation models in medical imaging lacks comprehensive benchmarks, standardized pipelines, and extensible libraries covering diverse FMs, tasks, usages, datasets, and sensitive attributes.

Approach

A standardized, extensible pipeline that evaluates many classification and segmentation FMs under multiple usage protocols (linear probing, PEFT, CLIP zero-shot/adapt, prompt-based SAM segmentation) with multiple group-fairness, predictive-alignment, and equity-scaled trade-off metrics, plus integrated bias mitigation methods.

Key results

• Average AUC delta across classification methods is larger than 5%, with SegVol and FT-SAM segmentation DSC delta reaching up to 10%.
• In the mitigation study on HAM10000 (SEG), SAMed + FEBS achieves the best DSC_Avg of 91.63 with DSC_Delta 0.02, while SAMed + InD worsens both utility and fairness (88.85 DSC, 1.42 delta).
• In the mitigation study on classification, LoRA + Resampling attains the highest AUC_Avg (83.27) but LoRA + LAFTR reduces AUC_Delta to 0.62 from 1.95 for LoRA.

Measured effects reported in the paper

Surprising findings

• Existing bias mitigation methods are not always effective for FMs; some (e.g., SAMed + InD) lower both utility and fairness.
• Simple LP achieves the best classification utility-equity tradeoff among the mitigation methods, outperforming LoRA-based variants on equity-scaled AUC.

Limitations

• Focuses only on binarized sensitive attributes and on classification and segmentation tasks.
• Mitigation evaluation limited to three datasets for classification and one for segmentation.
• Some models (e.g., 2D FMs on 3D data) rely on slice-wise 2.5D processing.

Experimental setup

Datasets: CheXpert MIMIC-CXR HAM10000 FairVLMed10k GF3300 PAPILA BRSET TUSC FairSeg Montgomery County X-ray KiTS 2023 CANDI

Benchmarks: CheXpert MIMIC-CXR HAM10000 FairVLMed10k GF3300 PAPILA BRSET TUSC FairSeg KiTS 2023 CANDI IRCADb

Metrics: AUC DSC AUC delta DSC delta equalized odds ECE delta equity-scaled AUC equity-scaled DSC demographic parity

Baselines: LP LoRA GroupDRO Resampling LAFTR SAMed FEBS InD

Builds on: SAM SAMed LoRA GroupDRO LAFTR FairCLIP

Closest papers by meaning

• The Boundaries of Fair AI in Medical Image Prognosis: A Causal Perspective (NeurIPS 2025 · similarity 0.91)
• FairSeg: A Large-Scale Medical Image Segmentation Dataset for Fairness Learning Using Segment Anything Model with Fair Error-Bound Scaling (ICLR 2024 · similarity 0.89)
• FairTune: Optimizing Parameter Efficient Fine Tuning for Fairness in Medical Image Analysis (ICLR 2024 · similarity 0.89)
• FairCLIP: Harnessing Fairness in Vision-Language Learning (CVPR 2024 · similarity 0.86)
• FairLLaVA: Fairness-Aware Parameter-Efficient Fine-Tuning for Large Vision-Language Assistants (CVPR 2026 · similarity 0.85)
• Distribution-aware Fairness Learning in Medical Image Segmentation From A Control-Theoretic Perspective (ICML 2025 · similarity 0.85)