Research map / Federated learning and privacy / Personalized and heterogeneous FL
FuseFL: One-Shot Federated Learning through the Lens of Causality with Progressive Model Fusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FuseFL, a one-shot federated learning method that splits networks into blocks and progressively trains, fuses, and freezes each block across clients to augment intermediate features and mitigate spurious fitting under data heterogeneity.
Problem
One-shot federated learning (single communication round) suffers a large performance gap versus normal multi-round FL and ensemble methods, due to locally isolated models fitting spurious correlations under non-IID data.
Approach
Causal analysis (SEM with invariant and spurious features) motivates progressively training each network block across clients and fusing intermediate features from other clients' blocks via lightweight adaptors, with reduced hidden width (ratio sqrt(M)) to keep storage near one model size.
Key results
- On CIFAR-10 with alpha=0.1, FuseFL (K=4) reaches 73.79% accuracy versus 57.5% for ensemble and 53.56% for FedDF-style... (best baseline DENSE 50.26%).
- FuseFL with K=2 on MNIST alpha=0.3 reaches 98.43% vs DENSE 76.48%.
- Memory with M=5: FuseFL K=2 uses 53.71MB vs 213.31MB for ensemble, while single model is 42.66MB.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace isolated one-shot training with progressive block fusion (K=2) | ResNet-18 one-shot federated learning · CIFAR-10 (α=0.5) | accuracy: 79.91 -> 84.34 | Helped |
| replace isolated one-shot training with progressive block fusion (K=2) | ResNet-18 one-shot federated learning · SVHN (α=0.5) | accuracy: 85.7 -> 90.87 | Helped |
| replace isolated one-shot training with progressive block fusion (K=4) | ResNet-18 one-shot federated learning · CIFAR-100 (α=0.5) | accuracy: 53.39 -> 49.30 | Hurt |
| replace isolated one-shot training with progressive block fusion (K=2) | ResNet-18 one-shot federated learning · Tiny-ImageNet (α=0.5) | accuracy: 45.8 -> 34.34 | Hurt |
| use average instead of convolution adaptor (ablation) | FuseFL · CIFAR-10 (α=0.1) | accuracy: 73.79 -> 68.08 | Hurt |
| use average instead of convolution adaptor (ablation) | FuseFL · CIFAR-10 (α=0.3) | accuracy: 84.58 -> 71.49 | Hurt |
| use average instead of convolution adaptor (ablation) | FuseFL · CIFAR-10 (α=0.5) | accuracy: 81.15 -> 80.35 | Hurt |
| use heterogeneous client architectures (ResNet-10/18/26) | FuseFL · CIFAR-10 (α=0.1, 0.3, 0.5) | accuracy: 73.79 -> 75.33; 84.58 -> 81.71; 81.15 -> 82.71 | Mixed |
| increase split blocks from K=2 to K=8 (ablation) | FuseFL · CIFAR-10 (M=5) | accuracy: 84.34 -> 74.99 | Hurt |
| increase split blocks from K=2 to K=8 (ablation) | FuseFL · CIFAR-10 (M=50) | accuracy: 42.18 -> 28.28 | Hurt |
| increase split blocks from K=2 to K=8 (ablation) | FuseFL · SVHN (M=50) | accuracy: 72.25 -> 42.19 | Hurt |
| use average feature fusion (ablation) | FuseFL · ResNet-18 memory (M=5) | memory: 53.71MB -> 53.32MB | Helped |
Surprising findings
- FuseFL outperforms the ensemble upper-bound on most settings, except CIFAR-100 (alpha 0.3, 0.5) and Tiny-ImageNet, which the authors attribute to larger class-level divergence.
Limitations
- Gains shrink on datasets with large class divergence such as CIFAR-100 and Tiny-ImageNet, where FuseFL falls below ensemble.
- Concat-and-freeze extension to transformers/LLMs is not evaluated.
- Requires K communication rounds (though each transmits only part of the model).
Experimental setup
Datasets: MNIST FMNIST CIFAR-10 SVHN CIFAR-100 Tiny-ImageNet
Benchmarks: MNIST FMNIST CIFAR-10 SVHN CIFAR-100 Tiny-ImageNet
Metrics: test accuracy memory occupation (MB)
Baselines: FedAvg FedDF Fed-DAFL Fed-ADI DENSE Ensemble
Builds on: FedAvg DENSE CCVR invariant risk minimization
Closest papers by meaning
- Revisiting Ensembling in One-Shot Federated Learning (NeurIPS 2024 · similarity 0.89)
- One-Shot Heterogeneous Federated Learning with Local Model-Guided Diffusion Models (ICML 2025 · similarity 0.89)
- Does One-shot Give the Best Shot? Mitigating Model Inconsistency in One-shot Federated Learning (ICML 2025 · similarity 0.89)
- SpaFL: Communication-Efficient Federated Learning With Sparse Models And Low Computational Overhead (NeurIPS 2024 · similarity 0.88)
- One-shot Federated Learning via Synthetic Distiller-Distillate Communication (NeurIPS 2024 · similarity 0.88)
- Convergence Analysis of Split Federated Learning on Heterogeneous Data (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.