Research map / Federated learning and privacy / Communication-efficient FL
Towards Straggler-Resilient Split Federated Learning: An Unbalanced Update Approach
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MU-SplitFed, a zeroth-order split federated learning method where the server performs tau unbalanced local updates per client round to decouple training progress from stragglers, with convergence O(sqrt(d/(tau T))) and experiments on image classification and OPT-1.3B.
Problem
Stragglers in split federated learning (SFL) stall synchronous training because the Split Server must wait for client activations, and the many communication rounds amplify delay.
Approach
The Split Server performs tau local ZO updates on its side model per client-server round (using the unperturbed embedding) while waiting for perturbed embeddings, combined with zeroth-order client updates; this reduces the number of communication rounds by a factor of tau and yields a linear speedup in convergence analysis.
Key results
- On CIFAR-100 with tau=2, MU-SplitFed reaches 32.16% test accuracy vs. 25.33% for GAS and 16.58% for vanilla SplitFed (tau=1).
- On OPT-1.3B, increasing tau for a fixed cut layer (L_c=4) reduces communication rounds to 85% accuracy by up to 33%.
- Client-side peak memory for OPT-1.3B fine-tuning drops to 1.05 GB vs. 8.02 GB for FedAvg and 5.64 GB for FedLoRA.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use MU-SplitFed (τ=2) instead of Vanilla SplitFed | AlexNet · CIFAR-10 test | accuracy: 69.73 -> 77.86 | Helped |
| use MU-SplitFed (τ=2) instead of Vanilla SplitFed | AlexNet · Fashion-MNIST test | accuracy: 77.50 -> 85.45 | Helped |
| use MU-SplitFed (τ=2) instead of Vanilla SplitFed | AlexNet · CINIC-10 test | accuracy: 51.96 -> 59.50 | Helped |
| use MU-SplitFed (τ=2) instead of Vanilla SplitFed | AlexNet · CIFAR-100 test | accuracy: 16.58 -> 32.16 | Helped |
| use MU-SplitFed instead of GAS | AlexNet · CIFAR-10, Fashion-MNIST, CINIC-10, and CIFAR-100 under straggler delays | accuracy over wall-clock time | Helped |
| increase server update ratio τ from 2 to 3 or 4 (ablation) | MU-SplitFed with AlexNet · CIFAR-10, Fashion-MNIST, CINIC-10, and CIFAR-100 test | accuracy: CIFAR-10: 77.86 -> 73.20, 69.40; Fashion-MNIST: 85.45 -> 85.28, 84.47; CINIC-10: 59.50 -> 55.75, 52.43; CIFAR-100: 32.16 | Hurt |
| increase τ at fixed cut layer Lc=4 (ablation) | MU-SplitFed on OPT-1.3B · communication rounds to 85% accuracy | communication-round reduction: up to 33% | Helped |
| decrease cut layer Lc while fixing τ (ablation) | MU-SplitFed on OPT-1.3B · convergence | accuracy | Helped |
| increase server-side client update ratio τ excessively at a fixed cut layer (ablation) | MU-SplitFed on OPT-1.3B · convergence | accuracy | Mixed |
| use MU-SplitFed instead of FedAvg | OPT-1.3B on SST-2 · client peak memory | peak memory: 8.02 GB -> 1.05 GB | Helped |
| use MU-SplitFed instead of FedLoRA | OPT-1.3B on SST-2 · client peak memory | peak memory: 5.64 GB -> 1.05 GB | Helped |
Surprising findings
- Accuracy drops for tau greater than 2 on the AlexNet experiments, which the authors attribute to the mismatch between tau and the cut layer.
Limitations
- Accuracy degrades when tau exceeds the value matched to the cut layer, so tau must be tuned jointly with the split point.
- Experiments are limited to small image models (AlexNet) and OPT-1.3B on SST-2; the LLM setting is only initial.
Experimental setup
Datasets: Fashion-MNIST CINIC-10 CIFAR-10 CIFAR-100 SST-2
Benchmarks: Fashion-MNIST CINIC-10 CIFAR-10 CIFAR-100 SST-2
Metrics: test accuracy communication rounds to 85% accuracy wall-clock accuracy peak client memory
Baselines: Vanilla SplitFed GAS FedAvg FedLoRA
Builds on: Split Federated Learning SPSA zeroth-order gradient estimation FedAvg GAS
Closest papers by meaning
- Convergence Analysis of Split Federated Learning on Heterogeneous Data (NeurIPS 2024 · similarity 0.90)
- FSL-SAGE: Accelerating Federated Split Learning via Smashed Activation Gradient Estimation (ICML 2025 · similarity 0.88)
- GAS: Generative Activation-Aided Asynchronous Split Federated Learning (AAAI 2025 · similarity 0.87)
- MultiSFL: Towards Accurate Split Federated Learning via Multi-Model Aggregation and Knowledge Replay (AAAI 2025 · similarity 0.86)
- FedWMSAM: Fast and Flat Federated Learning via Weighted Momentum and Sharpness-Aware Minimization (NeurIPS 2025 · similarity 0.86)
- Efficient Asynchronous Federated Learning with Prospective Momentum Aggregation and Fine-Grained Correction (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.