Research map / Federated learning and privacy / Personalized and heterogeneous FL
Ensemble Attention Distillation for Privacy-Preserving Federated Learning
Xuan Gong, Abhishek Sharma, Srikrishna Karanam, Ziyan Wu, Terrence Chen, David Doermann, Arun Innanje
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add attention ensemble distillation on top of logits distillation (ablation) | FedAD · CIFAR-10 (alpha=0.1) | accuracy: 72.67 -> 73.11 | Improved |
| use importance-weighted aggregation (Eq. 3) instead of existing aggregation schemes (ablation) | FedAD · CIFAR-10 (alpha=0.1) | accuracy: 65.73 -> 72.67 | Improved |
| set logits distillation temperature to infinity instead of tau=3 (ablation) | FedAD · CIFAR-10 (alpha=0.1) | accuracy: 67.81 -> 72.67 | Improved |
| set logits distillation temperature to infinity instead of tau=1 (ablation) | FedAD · Cityscapes | mIoU: 58.56 -> 71.86 | Improved |
| add attention distillation (ablation) | FedAD · Cityscapes | mIoU: 71.86 -> 72.97 | Improved |
| increase sample fraction gamma from 0.4 to 1 (ablation) | FedAD · CIFAR-10 (alpha=1) | accuracy: 82.48 -> 83.68 | Improved |
| increase sample fraction gamma from 0.4 to 1 (ablation) | FedAD · CIFAR-10 (alpha=0.1) | accuracy: 73.11 -> 73.40 | No clear effect |
| increase sample fraction gamma from 0.4 to 1 (ablation) | FedAD · CIFAR-10 communication | bandwidth (GB): 10.1 -> 27.6 | Hurt |
| use FedAD instead of FedDF | CIFAR-10 federated distillation · CIFAR-10 communication (gamma=0.4) | bandwidth (GB): 29.4 -> 10.1 | Improved |
| use FedAD instead of FedDF | CIFAR-100 federated distillation · CIFAR-100 (alpha=0.1) | accuracy: 39.33 -> 48.43 | Improved |
| use FedAD instead of FedMD on chest x-ray multi-label classification, distill C3+X3, test CXR14 | FedMD · CXR14 | mAUC: 70.97 -> 75.55 | Improved |
| use FedAD instead of FedMD, distill C3+X3, test CheXpert | FedMD · CheXpert | mAUC: 77.02 -> 82.65 | Improved |
Closest papers by meaning
- Federated Learning via Input-Output Collaborative Distillation (AAAI 2024 · similarity 0.93)
- Data-Free Black-Box Federated Learning via Zeroth-Order Gradient Estimation (AAAI 2025 · similarity 0.88)
- Little Is Enough: Boosting Privacy by Sharing Only Hard Labels in Federated Semi-Supervised Learning (AAAI 2025 · similarity 0.88)
- A New Federated Learning Framework Against Gradient Inversion Attacks (AAAI 2025 · similarity 0.88)
- Privacy-Preserving Federated Convex Optimization: Balancing Partial-Participation and Efficiency via Noise Cancellation (ICML 2025 · similarity 0.88)
- DictPFL: Efficient and Private Federated Learning on Encrypted Gradients (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
