Research map / Architectures and efficiency / Quantization

Understanding the Unfairness in Network Quantization

ICML 2025 · Architectures and efficiency › Quantization · approach: Post-training weight quantization

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Theoretically and empirically analyzes why post-training and quantization-aware training quantization worsen accuracy fairness across demographic groups, attributing it to group gradient norms and Hessian traces driven by class imbalance, and mitigates it with data augmentation.

Problem

Network quantization can exacerbate accuracy disparities across demographic groups, and it was unclear what factors cause this and how QAT and PTQ differ in their fairness impact.

Approach

Derives upper bounds on group excessive loss under PTQ and QAT in terms of group gradient norm and Hessian trace, shows both scale inversely with group size under class imbalance, and shows QAT adds extra interaction terms that make its unfairness worse; simple data augmentation (geometric transforms, random erasing) rebalances data to reduce the disparity.

Key results

• On UTK-Face with ResNet50, phi(D) rises from 2.4% at int16 to 18.8% at int4 under PTQ, and from 3.3% to 25.4% under QAT.
• QAT shows larger unfairness than PTQ at low bit-widths across datasets (e.g., FER2013 ResNet50 int4: 13.4% PTQ vs 44.4% QAT).
• Geometric transformation and random erasing augmentation reduce UTK-Face ResNet50 int4 phi(D) from 18.8% to 2.6% and 1.7% under PTQ.

Measured effects reported in the paper

Surprising findings

• QAT, despite preserving higher overall accuracy at low bit-widths, is less fair than PTQ.
• Quantization unfairness on CIFAR-10 increases only marginally (0.9% to 1.3% under PTQ), unlike the imbalanced UTK-Face dataset.

Limitations

• Analysis limited to image classification with ResNet and VGG models and weight-only quantization.
• Mitigation limited to two data augmentation methods; authors note solutions beyond augmentation are unexplored.
• Theoretical bounds are upper bounds with higher-order terms neglected; relationship is shown empirically rather than proven causal.

Experimental setup

Datasets: UTK-Face FER2013 CIFAR-10 MNIST

Benchmarks: UTK-Face FER2013 CIFAR-10 MNIST

Metrics: fairness metric phi(D) (max gap of excessive loss across gro per-group accuracy

Baselines: Fair Mixup Non-mitigation PTQ Non-mitigation QAT

Builds on: Post-Training Quantization Quantization-Aware Training Random Erasing Fair Mixup

Closest papers by meaning

• Effective Interplay between Sparsity and Quantization: From Theory to Practice (ICLR 2025 · similarity 0.86)
• QBB: Quantization with Binary Bases for LLMs (NeurIPS 2024 · similarity 0.85)
• JointSQ: Joint Sparsification-Quantization for Distributed Learning (CVPR 2024 · similarity 0.85)
• Scaling Laws for Floating–Point Quantization Training (ICML 2025 · similarity 0.85)
• Quantization Error Propagation: Revisiting Layer-Wise Post-Training Quantization (NeurIPS 2025 · similarity 0.85)
• LeanQuant: Accurate and Scalable Large Language Model Quantization with Loss-error-aware Grid (ICLR 2025 · similarity 0.85)