Research map / Federated learning and privacy / Personalized and heterogeneous FL
One-shot Federated Learning via Synthetic Distiller-Distillate Communication
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FedSD2C, a one-shot federated learning framework where clients send synthetic distillates (Core-Set selected by V-information, Fourier-perturbed, and optimized in a pre-trained autoencoder's latent space) instead of local models or generator-inverted data.
Problem
One-shot federated learning suffers accuracy loss; data-free knowledge distillation methods incur two-tier information loss and are hurt by data heterogeneity, degrading performance on complex datasets.
Approach
Clients distill local data end-to-end into compact latent distillates (V-information Core-Set selection, Fourier amplitude perturbation for privacy, optimization in a server-provided pretrained autoencoder's latent space) and share these with soft labels, rather than sharing inconsistent local models for generator-based inversion.
Key results
- Under extreme heterogeneity (alpha=0.1), FedSD2C reaches up to 2.6x the accuracy of the best baseline on Tiny-ImageNet (26.83 vs 10.29 for ResNet-18) and 1.8x on OpenImage.
- FedSD2C's communication cost is at most about 4% of sharing models (e.g., 0.5MB vs 44MB on ImageNette).
- Fourier perturbation at lambda=0.8 reduces PSNR/SSIM to 16.95/35.89 on Tiny-ImageNet while keeping accuracy close to no-privacy setting (26.83 vs 28.29 ResNet-18).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace V-information Core-Set selection with random selection (ablation) | FedSD2C · Tiny-ImageNet, ResNet-18, α=0.1, ipc=50 | accuracy: 26.83 -> 23.32 | Hurt |
| Replace V-information Core-Set selection with random selection (ablation) | FedSD2C · ImageNette, ResNet-18, α=0.1, ipc=50 | accuracy: 47.52 -> 42.06 | Hurt |
| Increase the synthetic data amount from ipc=50 to ipc=500 (ablation) | FedSD2C · CIFAR-10 α=0.1, ResNet-18 | accuracy: 44.72 -> 56.95 | Helped |
| Apply DP-SGD with finite ε (ablation) | FedSD2C · Tiny-ImageNet, ResNet-18, α=0.1, ipc=50 | accuracy: 26.83 -> 22.92 (ε=1); 26.83 -> 25.13 (ε=4); 26.83 -> 26.01 (ε=8) | Hurt |
| Replace Fourier perturbation with wavelet perturbation at λ=0.5 (ablation) | FedSD2C · Tiny-ImageNet, ResNet-18, α=0.1, ipc=50 | accuracy: 28.09 -> 26.91 | Hurt |
| Remove the pre-trained Autoencoder (ablation) | FedSD2C · ImageNette, ResNet-18, α=0.1 | accuracy: 47.52 -> 19.90 | Hurt |
| Use the proposed FedSD2C instead of the strongest baseline | One-shot FL · Tiny-ImageNet α=0.1, ResNet-18 | accuracy: 10.29 -> 26.83 | Helped |
| Use the proposed FedSD2C instead of the strongest baseline | One-shot FL · OpenImage, ConvNet | accuracy: 13.59 -> 23.00 | Helped |
Surprising findings
- Random-noise and FedMix privacy techniques degrade accuracy substantially, while FedSD2C's Fourier perturbation keeps accuracy close to the unprotected setting.
- Uploading the raw Core-Set directly yields the best accuracy (31.01 on Tiny-ImageNet), exceeding FedSD2C, but at higher communication and privacy cost.
Limitations
- Local distillation adds computational overhead on client devices compared with sharing models.
- Pre-trained autoencoders trained on natural images may be less effective on other domains such as medical images, requiring many more synthesis iterations (T_syn=1000).
- Performance on low-resolution CIFAR-10 at default ipc=50 is lower than baselines, requiring ipc=500 to be competitive.
Experimental setup
Datasets: Tiny-ImageNet ImageNette OpenImage ImageNet-1K CIFAR-10 COVID-FL
Benchmarks: Tiny-ImageNet ImageNette OpenImage CIFAR-10 COVID-FL
Metrics: top-1 accuracy PSNR SSIM communication cost (MB) TPR@FPR=0.1%
Baselines: FedAVG F-DAFL DENSE Co-Boosting
Builds on: data-free knowledge distillation DENSE Co-Boosting V-information DAFL
Closest papers by meaning
- Revisiting Ensembling in One-Shot Federated Learning (NeurIPS 2024 · similarity 0.91)
- FedBEns: One-Shot Federated Learning based on Bayesian Ensemble (ICML 2025 · similarity 0.90)
- One-Shot Heterogeneous Federated Learning with Local Model-Guided Diffusion Models (ICML 2025 · similarity 0.90)
- Efficient Federated Learning via Clients-to-Server Knowledge Distillation (Student Abstract) (AAAI 2025 · similarity 0.89)
- FedLPA: One-shot Federated Learning with Layer-Wise Posterior Aggregation (NeurIPS 2024 · similarity 0.89)
- Low Precision Local Training is Enough for Federated Learning (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.