Research map / Federated learning and privacy / Communication-efficient FL
FedEL: Federated Elastic Learning for Heterogeneous Devices
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FedEL, a federated elastic learning framework that combines a sliding-window block selection with ElasticTrainer-style tensor selection and a local/global tensor-importance adjustment to speed up heterogeneous federated training.
Problem
Stragglers with heterogeneous hardware prolong federated learning rounds; existing fixes (client selection, asynchronous FL, partial width/depth training) lose accuracy or cause stale updates, and naively applying ElasticTrainer in FL restricts training to back layers and amplifies local model drift
Approach
A window slides over model blocks so that every client (including stragglers) progressively trains all parts of the DNN within a shared runtime budget, with ElasticTrainer selecting important tensors inside the window; local tensor importance is blended with global importance estimated from consecutive global models.
Key results
- FedEL reaches up to 3.87x time-to-accuracy speedup over FedAvg on the speech task (100 devices) while matching or exceeding accuracy (58.26% vs 58.04%).
- On the 10-device Jetson testbed with CIFAR10, FedEL achieves 1.87x speedup over FedAvg with 56.51% accuracy vs 56.13%.
- FedEL reduces memory usage by up to 32.7% and average energy use by 49.59% relative to FedAvg.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FedEL instead of FedAvg | CIFAR10, 10 devices · CIFAR10 | training speedup: 1.87× | Helped |
| Use FedEL instead of FedAvg | Tiny ImageNet, 100 devices · Tiny ImageNet | training speedup: 3.59× | Helped |
| Use FedEL instead of FedAvg | Google Speech, 100 devices · Google Speech | training speedup: 3.87× | Helped |
| Use FedEL instead of FedAvg | Reddit, 100 devices · Reddit | training speedup: 3.13× | Helped |
| Replace FedAvg with FedEL | CIFAR10, 10 devices · CIFAR10 | accuracy: 56.13% -> 56.51% | No clear effect |
| Replace FedAvg with FedEL | Tiny ImageNet, 100 devices · Tiny ImageNet | accuracy: 33.76% -> 34.96% | Helped |
| Replace FedAvg with FedEL | Google Speech, 100 devices · Google Speech | accuracy: 58.04% -> 58.26% | No clear effect |
| Replace FedAvg with FedEL | Reddit, 100 devices · Reddit | perplexity: 77.48 -> 77.23 | Helped |
| Replace FedAvg with ElasticTrainer | federated learning across four datasets · four FL datasets | accuracy: over 28.6% accuracy loss | Hurt |
| Use moderate β (0.4 or 0.6) instead of β = 1 or β = 0 (ablation) | FedEL · CIFAR10 and three 100-device tasks | accuracy | Mixed |
| Reduce runtime threshold T_th (ablation) | FedEL · CIFAR10 and three 100-device tasks | convergence speed | Hurt |
| Move the end edge directly to the current front edge (FedEL-C) (ablation) | FedEL · CIFAR10 | accuracy | Hurt |
Surprising findings
- Directly deploying ElasticTrainer in FL cuts training time but loses about 16 accuracy points on CIFAR10 (40.03% vs 56.13% for FedAvg), showing naive single-device selection fails under non-iid FL.
Limitations
- Experiments use only two types of edge devices (ten Jetson kits), so the large-scale simulation may not capture more extreme heterogeneity.
- Network bandwidth variation is not modeled.
- Training time deviates from the target threshold by up to 18.9% for the ALBERT model.
Experimental setup
Datasets: CIFAR10 Tiny ImageNet Google Speech Commands Reddit
Benchmarks: CIFAR10 Tiny ImageNet Google Speech Commands Reddit
Metrics: time-to-accuracy test accuracy perplexity training time memory usage energy consumption
Baselines: FedAvg ElasticTrainer HeteroFL DepthFL PyramidFL TimelyFL FIARSE
Builds on: ElasticTrainer FedAvg FedProx FedNova
Closest papers by meaning
- FedCompass: Efficient Cross-Silo Federated Learning on Heterogeneous Client Devices Using a Computing Power-Aware Scheduler (ICLR 2024 · similarity 0.88)
- FedQS: Optimizing Gradient and Model Aggregation for Semi-Asynchronous Federated Learning (NeurIPS 2025 · similarity 0.88)
- Low Precision Local Training is Enough for Federated Learning (NeurIPS 2024 · similarity 0.88)
- TurboSVM-FL: Boosting Federated Learning through SVM Aggregation for Lazy Clients (AAAI 2024 · similarity 0.87)
- Fed-QSSL: A Framework for Personalized Federated Learning under Bitwidth and Data Heterogeneity (AAAI 2024 · similarity 0.87)
- FedLPS: Heterogeneous Federated Learning for Multiple Tasks with Local Parameter Sharing (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.