Research map / Federated learning and privacy / Personalized and heterogeneous FL
Bridging Model Heterogeneity in Federated Learning via Uncertainty-based Asymmetrical Reciprocity Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FedType, which trains small identical ResNet-18 proxy models that are the only thing shared with the server, and uses uncertainty-based asymmetrical reciprocity learning (conformal prediction sets) to bidirectionally distill between proxies and large private client models, without public data.
Problem
Model-heterogeneous federated learning typically relies on public data and exchanges sensitive side information or full client models, raising privacy, security, and communication concerns.
Approach
Clients share only a small identical proxy model; private and proxy models are trained with forward KD (proxy from private) and a backward ranking-based imitation loss that transfers only high-confidence labels, selected via dynamic conformal prediction sets and weighted by a consensus score between the two models' set sizes.
Key results
- On CIFAR-10 with alpha=0.5 and FedAvg, FedType private models reach 86.83% average client accuracy versus 63.39% for the global model and 82.57% for proxies, outperforming FML (85.24%) and FedProto (78.17%) under the heterogeneous setting.
- Under the CIFAR-100 alpha=0.1 setting with pFedBayes, FedType private models reach 69.60% versus 68.74% for FML.
- The ablation shows that replacing the backward loss with symmetric KD drops CIFAR-10 accuracy from 86.83% to 82.24%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use asymmetric backward loss instead of traditional symmetric KD (ablation) | FedType · CIFAR-10 | accuracy: 82.24 -> 86.83 | Helped |
| use asymmetric backward loss instead of traditional symmetric KD (ablation) | FedType · Fed-ISIC19 | accuracy: 49.89 -> 55.95 | Helped |
| use conformal prediction sets instead of fixed top K=3 labels (ablation) | FedType · CIFAR-10 | accuracy: 84.69 -> 86.83 | Helped |
| use conformal prediction sets instead of fixed top K=3 labels (ablation) | FedType · Fed-ISIC19 | accuracy: 51.26 -> 55.95 | Helped |
| estimate sample consensus weights instead of fixing η=1 (ablation) | FedType · CIFAR-10 | accuracy: 85.12 -> 86.83 | Helped |
| estimate sample consensus weights instead of fixing η=1 (ablation) | FedType · Fed-ISIC19 | accuracy: 52.40 -> 55.95 | Helped |
| dynamically adjust calibration g with performance instead of fixing g=0.5 (ablation) | FedType · CIFAR-10 | accuracy: 86.24 -> 86.83 | Helped |
| dynamically adjust calibration g with performance instead of fixing g=0.5 (ablation) | FedType · Fed-ISIC19 | accuracy: 54.17 -> 55.95 | Helped |
| increase the proxy model architecture similarity to the private model family (ablation) | FedType · CIFAR-10 | accuracy | Helped |
| adjust calibration g in response to accuracy fluctuations rather than use constant g₂ (ablation) | FedType · CIFAR-10 and Fed-ISIC19 | accuracy | Helped |
Surprising findings
- The shared global model performs much worse than the individual proxy and private models, especially on harder non-IID settings (e.g., 38.36% vs 56.24% on CIFAR-100 with alpha=1 under FedAvg).
Limitations
- Selecting the proxy model is non-trivial and strongly affects performance.
- The uncertainty-based asymmetrical reciprocity learning runs slower than naive bidirectional distillation.
Experimental setup
Datasets: FMNIST CIFAR-10 CIFAR-100 Fed-ISIC19 AG News
Benchmarks: FMNIST CIFAR-10 CIFAR-100 Fed-ISIC19 AG News
Metrics: average client accuracy
Baselines: FedProto FML FedAvg FedProx pFedMe pFedBayes Homo-VGG-19
Builds on: FML RAPS FedAvg FedProx pFedMe
Closest papers by meaning
- A Bayesian Approach for Personalized Federated Learning in Heterogeneous Settings (NeurIPS 2024 · similarity 0.90)
- FedP3: Federated Personalized and Privacy-friendly Network Pruning under Model Heterogeneity (ICLR 2024 · similarity 0.90)
- FedLMT: Tackling System Heterogeneity of Federated Learning via Low-Rank Model Training with Theoretical Guarantees (ICML 2024 · similarity 0.89)
- Fake It Till Make It: Federated Learning with Consensus-Oriented Generation (ICLR 2024 · similarity 0.89)
- FedGPS: Statistical Rectification Against Data Heterogeneity in Federated Learning (NeurIPS 2025 · similarity 0.89)
- FedCompetitors: Harmonious Collaboration in Federated Learning with Competing Participants (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.