Research map / Federated learning and privacy / Personalized and heterogeneous FL

Unlocking the Potential of Model Calibration in Federated Learning

ICLR 2025 · Federated learning and privacy › Personalized and heterogeneous FL · approach: Federated learning with pretrained models · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes NUCFL, a federated learning framework that adds an auxiliary calibration penalty (DCA/MDCA) to client training with a per-client weight set by the similarity between local and global model updates, improving calibration (ECE/SCE) without hurting accuracy.

Problem

Federated learning models are often miscalibrated (overconfident) under non-IID data, and post-hoc calibration needs a holdout set that privacy constraints often rule out.

Approach

Dynamically weight each client's auxiliary calibration penalty by the similarity (cosine or CKA) between its local update and the global model update, so clients resembling the global model get a larger calibration penalty; this works with any FL algorithm and any auxiliary calibration loss.

Key results

• On CIFAR-100 non-IID FedAvg, NUCFL (DCA+L-CKA) reaches 62.05% accuracy and 6.14% ECE versus 61.34% / 10.52% for uncalibrated FedAvg and 61.24% / 7.69% for DCA.
• On FEMNIST, NUCFL (MDCA+L-CKA) with FedAvg achieves 3.28% ECE versus 3.75% for MDCA alone.
• Across the five FL algorithms, NUCFL variants reduce ECE relative to DCA/MDCA on CIFAR-100 while keeping accuracy comparable.

Measured effects reported in the paper

Surprising findings

• Reversing the similarity weighting (penalizing clients less similar to the global model more) degrades calibration below uncalibrated FL.
• Standard train-time calibration methods such as focal loss, label smoothing, and MMCE often increase calibration error when applied to non-IID FL, while DCA and MDCA are more robust.

Limitations

• Theoretical guarantees for calibration are not provided; the work is purely empirical.
• Evaluated only on image classification with small CNN, AlexNet, and ResNet-34 backbones and 50 clients.
• Relies on the assumption that similarity between local and global updates indicates representativeness of global calibration needs.

Experimental setup

Datasets: MNIST FEMNIST CIFAR-10 CIFAR-100

Benchmarks: MNIST FEMNIST CIFAR-10 CIFAR-100

Metrics: accuracy ECE SCE

Baselines: Uncal. cross-entropy Focal loss Label smoothing Brier score MMCE FLSD MbLS DCA MDCA FedCal Temperature scaling

Builds on: FedAvg FedProx Scaffold FedDyn FedNova DCA MDCA temperature scaling

Closest papers by meaning

• FedCal: Achieving Local and Global Calibration in Federated Learning via Aggregated Parameterized Scaler (ICML 2024 · similarity 0.93)
• Calibrated One Round Federated Learning with Bayesian Inference in the Predictive Space (AAAI 2024 · similarity 0.89)
• Federated Learning with Sample-level Client Drift Mitigation (AAAI 2025 · similarity 0.88)
• FedGPS: Statistical Rectification Against Data Heterogeneity in Federated Learning (NeurIPS 2025 · similarity 0.88)
• Watch Your Head: Assembling Projection Heads to Save the Reliability of Federated Models (AAAI 2024 · similarity 0.87)
• FLHetBench: Benchmarking Device and State Heterogeneity in Federated Learning (CVPR 2024 · similarity 0.87)