Research map / Federated learning and privacy / Personalized and heterogeneous FL
FedUV: Uniformity and Variance for Heterogeneous Federated Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FedUV, a local-training regularizer for federated learning that combines classifier probability-variance hinge loss with hyperspherical uniformity of encoder representations to emulate IID training under label and feature shift.
Problem
Federated learning performance degrades under non-IID data, with the final classifier layer most prone to local bias; existing regularizers rely on the global model and add computation or memory cost.
Approach
Instead of regularizing toward a global model, the method penalizes local training when the classifier's batch-wise probability variance per class falls below an IID-equivalent threshold and when encoder representations are not uniformly spread on the hypersphere (RBF-kernel Gaussian energy), requiring no extra forward passes or weight comparisons.
Key results
- On STL-10 with alpha=0.01, FedUV reaches 30.4% accuracy vs 27.6% for FedAvg and 26.0% for MOON.
- On Office-Home (feature shift), FedUV reaches 45.4% vs 42.2% for FedAvg at the default setting.
- On HAM10000 feature-shift setting, FedUV achieves 65.9% vs 61.9% for FedAvg; on CIFAR-100 (alpha=0.01) 55.7% vs 51.6%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add classifier variance regularization (ablation) | FedUV · STL-10 (α=0.01) | accuracy: 24.1±1.3 -> 30.4±1.4 | Helped |
| add classifier variance regularization (ablation) | FedUV · CIFAR-100 (α=0.01) | accuracy: 52.8±1.2 -> 55.7±1.0 | Helped |
| add classifier variance regularization (ablation) | FedUV · Tiny ImageNet (α=0.01) | accuracy: 37.9±1.7 -> 40.3±0.8 | Helped |
| add classifier variance regularization (ablation) | FedUV · PACS | accuracy: 63.7±0.3 -> 65.9±0.9 | Helped |
| add classifier variance regularization (ablation) | FedUV · HAM10000 | accuracy: 73.4±0.2 -> 73.9±0.5 | Helped |
| add classifier variance regularization (ablation) | FedUV · Office-Home | accuracy: 43.5±1.3 -> 45.4±1.0 | Helped |
| add hyperspherical uniformity regularization (ablation) | FedUV · STL-10 (α=0.01) | accuracy: 29.5±1.4 -> 30.4±1.4 | Helped |
| add hyperspherical uniformity regularization (ablation) | FedUV · CIFAR-100 (α=0.01) | accuracy: 54.0±1.1 -> 55.7±1.0 | Helped |
| add hyperspherical uniformity regularization (ablation) | FedUV · Tiny ImageNet (α=0.01) | accuracy: 38.8±1.5 -> 40.3±0.8 | Helped |
| add hyperspherical uniformity regularization (ablation) | FedUV · PACS | accuracy: 60.1±0.3 -> 65.9±0.9 | Helped |
| add hyperspherical uniformity regularization (ablation) | FedUV · HAM10000 | accuracy: 72.5±0.4 -> 73.9±0.5 | Helped |
| add hyperspherical uniformity regularization (ablation) | FedUV · Office-Home | accuracy: 41.8±1.1 -> 45.4±1.0 | Helped |
Surprising findings
- Freezing the classifier (Freeze) performs best on STL-10 with alpha=1.0 (72.1%), outperforming FedUV (68.5%).
- Variance regularization alone does not significantly help feature-shift settings, while uniformity alone helps there.
Limitations
- Performance degrades significantly with high local epochs (E=40), unlike MOON and FedProx.
- Convergence proof is not established because the hinge loss on variance is non-smooth.
- Hyperparameters tuned on CIFAR-10 and fixed across datasets; the sigma choice relies on median heuristic.
Experimental setup
Datasets: STL-10 CIFAR-100 Tiny ImageNet PACS HAM10000 Office-Home CIFAR-10
Benchmarks: STL-10 CIFAR-100 Tiny ImageNet PACS HAM10000 Office-Home
Metrics: test accuracy wall-clock time per aggregation round
Baselines: FedAvg FedProx MOON Freeze (FedBABU-style frozen classifier)
Builds on: FedAvg FedBABU MOON FedProx CCVR
Closest papers by meaning
- Accelerating Heterogeneous Federated Learning with Closed-form Classifiers (ICML 2024 · similarity 0.88)
- FedLoGe: Joint Local and Generic Federated Learning under Long-tailed Data (ICLR 2024 · similarity 0.88)
- Widening the Network Mitigates the Impact of Data Heterogeneity on FedAvg (ICML 2025 · similarity 0.88)
- Uncertainty-Based Extensible Codebook for Discrete Federated Learning in Heterogeneous Data Silos (ICML 2025 · similarity 0.87)
- FedDAT: An Approach for Foundation Model Finetuning in Multi-Modal Heterogeneous Federated Learning (AAAI 2024 · similarity 0.87)
- FedAdaVR: Adaptive Variance Reduction for Robust Federated Learning under Limited Client Participation (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.