Research map / Federated learning and privacy / Personalized and heterogeneous FL
Ensemble Distillation for Robust Model Fusion in Federated Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use server-side ensemble distillation, initialized from the current-round averaged client model, instead of parameter averaging alone | ResNet-8 federated learning · CIFAR-10, α=1, target 80% accuracy | communication rounds to target accuracy: 78 ± 28 -> 20 ± 4 | Helped |
| Use server-side ensemble distillation, initialized from the current-round averaged client model, instead of parameter averaging alone | ResNet-8 federated learning · CIFAR-10, α=0.1, target 75% accuracy | communication rounds to target accuracy: 224 ± 38 -> 39 ± 10 | Helped |
| Use FedDF with batch normalization rather than FedAvg with group normalization | ResNet-8 federated learning · CIFAR-10, α=1 | top-1 test accuracy: 78.57 ± 0.22 -> 80.69 ± 0.43 | Helped |
| Use FedDF with batch normalization rather than FedAvg with group normalization | ResNet-8 federated learning · CIFAR-100, α=0.1 | top-1 test accuracy: 36.72 ± 1.50 -> 39.33 ± 0.03 | Helped |
| Initialize distillation from the current-round averaged model instead of the previous-round fused model (ablation) | FedDF with ResNet-8 · CIFAR-10, α=1, 40 local epochs | top-1 test accuracy: 74.13 ± 0.91 -> 80.43 ± 0.37 | Helped |
| Initialize distillation from the current-round averaged model instead of the previous-round fused model (ablation) | FedDF with ResNet-8 · CIFAR-10, α=0.1, 40 local epochs | top-1 test accuracy: 62.94 ± 1.12 -> 71.84 ± 0.86 | Helped |
| Replace Adam with SGD for server-side ensemble distillation (ablation) | FedDF with ResNet-8 · CIFAR-10, α=1 | top-1 test accuracy: 80.27 -> 76.68 | Hurt |
| Replace Adam with SGD for server-side ensemble distillation (ablation) | FedDF with ResNet-8 · CIFAR-10, α=0.1 | top-1 test accuracy: 71.52 -> 57.33 | Hurt |
| Use Adam instead of SGD for local client training (ablation) | FedDF with ResNet-8 · CIFAR-10, α=1 | top-1 test accuracy: 80.27 -> 83.32 | Helped |
| Use Adam instead of SGD for local client training (ablation) | FedDF with ResNet-8 · CIFAR-10, α=0.1 | top-1 test accuracy: 71.52 -> 72.58 | No clear effect |
| Add a proximal penalty to local client training (ablation) | FedDF with ResNet-8 · CIFAR-10, α=1 | top-1 test accuracy: 80.27 -> 80.56 | No clear effect |
| Add a proximal penalty to local client training (ablation) | FedDF with ResNet-8 · CIFAR-10, α=0.1 | top-1 test accuracy: 71.52 -> 71.64 | No clear effect |
Closest papers by meaning
- Feature Distillation is the Better Choice for Model-Heterogeneous Federated Learning (NeurIPS 2025 · similarity 0.86)
- An Aggregation-Free Federated Learning for Tackling Data Heterogeneity (CVPR 2024 · similarity 0.84)
- Adaptive Latent-Space Constraints in Personalized Federated Learning (NeurIPS 2025 · similarity 0.84)
- One-shot Federated Learning via Synthetic Distiller-Distillate Communication (NeurIPS 2024 · similarity 0.84)
- Personalized Federated Learning with Mixture of Models for Adaptive Prediction and Model Fine-Tuning (NeurIPS 2024 · similarity 0.83)
- Data-Free Black-Box Federated Learning via Zeroth-Order Gradient Estimation (AAAI 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.