Research map / Learning theory and algorithms / Fairness
On The Fairness Impacts of Hardware Selection in Machine Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that GPU hardware choice alone changes per-group accuracy and fairness of trained CNNs, attributes this to group gradient norms and loss-surface curvature, and proposes a distance-to-decision-boundary regularizer that reduces hardware-induced fairness violation.
Problem
Hardware selection (GPU type) is usually ignored, yet identical training runs on different hardware can produce different accuracies, and minority groups are disproportionately affected, which threatens fairness in ML-as-a-service settings.
Approach
Derives a second-order upper bound on hardware sensitivity per group, showing it depends on group gradient norms and group Hessian max eigenvalues; links the Hessian term to the average distance of group samples from the decision boundary, and mitigates by adding a penalty that aligns group and batch-wide boundary distances to the training loss.
Key results
- On imbalanced CIFAR-10 the mitigation (lambda=6e-5) reduces fairness violation from 1.91 to 0.77.
- On CelebA with ResNet50, the mitigation reduces fairness violation from 2.34 to 0.936.
- On UTKFace the minority class Others has hardware sensitivity 0.122 versus 0.034 for the majority White class, and the authors report roughly a threefold reduction in unfairness after mitigation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| reduce CIFAR-10 Ship training examples to 20% of the original size | CIFAR-10 ResNet34 · CIFAR-10 Ship | hardware sensitivity: 0.017 to 0.046 | Hurt |
| reduce CIFAR-10 Ship training examples to 20% of the original size | CIFAR-10 ResNet34 · CIFAR-10 Ship | gradient norm: 0.015 to 0.041 | Hurt |
| use decision-boundary distance disparity penalty (λ=6e-5) | CIFAR-10 imbalanced ResNet34 · CIFAR-10 imbalanced | fairness violation: 1.91 to 0.77 | Helped |
| use decision-boundary distance disparity penalty | CelebA ResNet50 · CelebA | fairness violation: 2.34 to 0.936 | Helped |
| use decision-boundary distance disparity penalty (λ=2e-5) | UTKFace · UTKFace | fairness violation | Helped |
| use decision-boundary distance disparity penalty | UTKFace · UTKFace Asian | accuracy difference across hardware | Hurt |
| increase the penalty from λ=1e-6 to λ=6e-5 | CIFAR-10 · CIFAR-10 | hardware sensitivity | Mixed |
Surprising findings
- Hardware sensitivity and unfairness increase with model complexity, with larger architectures showing more hardware-induced disparity.
Limitations
- Mitigation works via a grid search over lambda, with no principled tuning; results vary substantially across lambda values and can increase sensitivity for some classes.
- Experiments are limited to small CNNs/ResNets on three image datasets and four GPU types, and the authors note that training was limited by the slowest GPU.
- Mixed-precision training is used for CelebA, which confounds the hardware-only comparison.
Experimental setup
Datasets: CIFAR-10 UTKFace CelebA
Benchmarks: CIFAR-10 (imbalanced variant) UTKFace CelebA
Metrics: hardware sensitivity fairness violation (max hardware loss/accuracy difference ac class accuracy gradient norm Hessian max eigenvalue
Baselines: unmitigated training (no fairness penalty)
Builds on: differential privacy disparate impact analysis Taylor-expansion loss change bounds one-cycle learning rate
Closest papers by meaning
- FairBatch: Batch Selection for Model Fairness (ICLR 2021 · similarity 0.85)
- Understanding Fairness and Prediction Error through Subspace Decomposition and Influence Analysis (NeurIPS 2025 · similarity 0.84)
- The Disparate Benefits of Deep Ensembles (ICML 2025 · similarity 0.84)
- Understanding challenges to the interpretation of disaggregated evaluations of algorithmic fairness (NeurIPS 2025 · similarity 0.84)
- Fairness under Competition (NeurIPS 2025 · similarity 0.84)
- The Fragility of Fairness: Causal Sensitivity Analysis for Fair Machine Learning (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.