Research map / Federated learning and privacy / Communication-efficient FL
FSL-SAGE: Accelerating Federated Split Learning via Smashed Activation Gradient Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FSL-SAGE, a federated split learning method where clients train with gradient estimates from periodically aligned auxiliary models that emulate the server, with convergence analysis and experiments on ResNet-18/CIFAR and GPT2-medium/E2E.
Problem
Federated split learning for large models is communication-heavy and sequential in SL, while auxiliary-model (local loss) variants lack server feedback and can be inaccurate, so clients never receive true server-side gradient signal.
Approach
Clients use auxiliary models to estimate the cut-layer gradients returned by the server; these auxiliary models are periodically aligned (every l rounds) to the server-side model using stored smashed activations and true server gradients, giving server feedback at low communication cost. Also a lazy variant freezes alignment after T' rounds.
Key results
- On CIFAR-10 non-iid, FSL-SAGE reaches 82.75% vs 81.48% for SplitFedv1 (second best) and 42.72% for FedAvg (ResNet-18, 200 GiB budget).
- On CIFAR-100 iid, FSL-SAGE achieves 56.06% vs 50.94% for CSE-FSL.
- For a given accuracy (~81% on CIFAR-10), FSL-SAGE needs about 2.2x less communication than CSE-FSL and 10x less than SplitFed-SS.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FSL-SAGE instead of the strongest baseline, SplitFedv1 | ResNet-18 · CIFAR-10 iid | test accuracy (%): 84.21 -> 85.71 | Helped |
| Use FSL-SAGE instead of the strongest baseline, SplitFedv1 | ResNet-18 · CIFAR-10 non-iid | test accuracy (%): 81.48 -> 82.75 | Helped |
| Use FSL-SAGE instead of the strongest baseline, CSE-FSL | ResNet-18 · CIFAR-100 iid | test accuracy (%): 50.94 -> 56.06 | Helped |
| Use FSL-SAGE instead of the strongest baseline, SplitFedv1 | ResNet-18 · CIFAR-100 non-iid | test accuracy (%): 51.03 -> 51.90 | Helped |
| Use FSL-SAGE instead of CSE-FSL | ResNet-18 · CIFAR-10 at 81% test accuracy | communication cost: almost 2.2× lesser communication cost | Helped |
| Use FSL-SAGE instead of SplitFed-SS | ResNet-18 · CIFAR-10 at 81% test accuracy | communication cost: 10× lesser | Helped |
| Use FSL-SAGE instead of CSE-FSL | GPT2-medium LoRA fine-tuning · E2E test data | masked cross-entropy loss | Helped |
| Use FSL-SAGE instead of FedAvg | GPT2-medium LoRA fine-tuning · E2E test data | masked cross-entropy loss convergence speed | Hurt |
Surprising findings
- On GPT2-medium E2E, FedAvg converges fastest because communicating the model once per round uses fewer bytes than frequently transmitting smashed data.
Limitations
- Convergence analysis lacks linear speedup with number of clients because the server-side model is processed sequentially.
- Assumes the server has unbounded storage for the growing alignment dataset (simplified analysis).
- Lazy variant trades final accuracy for communication; with T'=0 the bound diverges.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 E2E
Benchmarks: CIFAR-10 CIFAR-100 E2E
Metrics: test accuracy communication cost (GiB) masked cross-entropy loss
Baselines: FedAvg SplitFedv1 SplitFedv2 CSE-FSL
Builds on: FedAvg SplitFed CSE-FSL LoRA
Closest papers by meaning
- Convergence Analysis of Split Federated Learning on Heterogeneous Data (NeurIPS 2024 · similarity 0.90)
- GAS: Generative Activation-Aided Asynchronous Split Federated Learning (AAAI 2025 · similarity 0.89)
- Towards Straggler-Resilient Split Federated Learning: An Unbalanced Update Approach (NeurIPS 2025 · similarity 0.88)
- FedQS: Optimizing Gradient and Model Aggregation for Semi-Asynchronous Federated Learning (NeurIPS 2025 · similarity 0.87)
- Revisiting Ensembling in One-Shot Federated Learning (NeurIPS 2024 · similarity 0.87)
- TurboSVM-FL: Boosting Federated Learning through SVM Aggregation for Lazy Clients (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.