Research map / Federated learning and privacy / Personalized and heterogeneous FL
FedCal: Achieving Local and Global Calibration in Federated Learning via Aggregated Parameterized Scaler
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FedCal, a post-hoc federated calibration method that trains order-preserving MLP scalers on each client and aggregates them by weight averaging (after weight matching) into a global scaler, achieving local and global calibration without a global validation set.
Problem
Model calibration in federated learning degrades under label-skewed (non-IID) data, and existing calibration methods either need a global validation set or fail to produce well-calibrated global models.
Approach
Train client-specific order-preserving MLP scalers for local calibration, align them with weight matching (permutation symmetry), and average their parameters to obtain a global scaler that minimizes global calibration error without any global validation data.
Key results
- Reduces global ECE by 47.66% on average versus the best non-validation baseline (Ens.) and 63.06% versus uncalibrated FL.
- On MNIST-CNN with beta=0.1, FedCal achieves 1.35% global ECE versus 2.13% for Ens. and 4.6% uncalibrated.
- Adding FedCal to FedProx on MNIST-CNN lowers global ECE from 3.19% to 1.07% at beta=0.1 with similar accuracy.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FedCal's personalized MLP scalers and aggregated global scaler instead of uncalibrated FL | FedAvg · Four benchmark datasets, varying β | global ECE: 63.06% | Helped |
| Use FedCal instead of the strongest baseline without a global validation set | Federated calibration · Four benchmark datasets, varying β | global ECE: 47.66% | Helped |
| Add the order-preserving constraint (ablation) | FedCal · MNIST-CNN, β=0.5 | top-3 accuracy: 87.91\% -> 92.3\% | Helped |
| Add the order-preserving constraint (ablation) | FedCal · MNIST-CNN, β=0.5 | global ECE: 1.32\% -> 1.71\% | Hurt |
| Add weight matching before aggregating scalers (ablation) | FedCal · MNIST, increasing non-IIDness | global ECE | Helped |
| Increase the MLP scaler size from 8 to 64 neurons (ablation) | FedCal · MNIST, increasing non-IIDness | global ECE | Helped |
| Add FedCal to FedProx (ablation) | FedProx · MNIST-CNN, β=1 | global ECE: 0.38 -> 0.40 | Hurt |
| Add FedCal to FedProx (ablation) | FedProx · MNIST-CNN, β=0.5, 0.3, 0.1 | global ECE: 0.57 -> 0.42; 1.32 -> 0.58; 3.19 -> 1.07 | Helped |
Surprising findings
- Ensembling and averaged-temperature scalers (Ens., AvgT) can be worse than uncalibrated models under strong non-IIDness, e.g. AvgT reaches 32% ECE for SVHN at beta=0.1 only for LR-TS, showing baseline aggregation instability.
- Theory suggests global calibration error stays asymptotically lower bounded under label skew even with bounded label-distribution divergence.
Limitations
- Theoretical bound relies on strong assumptions (feature variance bound, binary reduction, convexity) and is only sketched in the main text.
- Scaler increases communication overhead and requires weight matching computation; privacy techniques like HE/DP are not explored.
- Only label-skew non-IID is studied; a global-to-local trade-off with order-preservation is noted as incurring a small ECE increase.
Experimental setup
Datasets: MNIST SVHN CIFAR-10 CIFAR-100
Benchmarks: MNIST SVHN CIFAR-10 CIFAR-100
Metrics: Expected Calibration Error (global) test accuracy top-3 accuracy local ECE
Baselines: Uncalibrated FL Validation temperature scaling (Val. TS) Deep Ensemble averaging (Ens.) Averaged temperature (AvgT) LR-TS (MD-TS adapted to FL) FedProx
Builds on: FedAvg temperature scaling MD-TS Deep Ensembles order-preserving networks weight matching
Closest papers by meaning
- Unlocking the Potential of Model Calibration in Federated Learning (ICLR 2025 · similarity 0.93)
- Calibrated One Round Federated Learning with Bayesian Inference in the Predictive Space (AAAI 2024 · similarity 0.90)
- Locally Estimated Global Perturbations are Better than Local Perturbations for Federated Sharpness-aware Minimization (ICML 2024 · similarity 0.88)
- FedWMSAM: Fast and Flat Federated Learning via Weighted Momentum and Sharpness-Aware Minimization (NeurIPS 2025 · similarity 0.88)
- FedAdaVR: Adaptive Variance Reduction for Robust Federated Learning under Limited Client Participation (NeurIPS 2026 · similarity 0.87)
- FedQS: Optimizing Gradient and Model Aggregation for Semi-Asynchronous Federated Learning (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.