Research map / Federated learning and privacy / Personalized and heterogeneous FL
FIARSE: Model-Heterogeneous Federated Learning via Importance-Aware Submodel Extraction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FIARSE, a model-heterogeneous federated learning method that extracts client submodels by thresholding parameter magnitudes (importance implicitly encoded in values) with a threshold-controlled biased gradient descent, and proves O(1/sqrt(T)) convergence under partial participation.
Problem
Model-heterogeneous federated learning where clients have varying computation capacities and must train submodels of the global model; existing static or rolling submodel extraction ignores parameter importance, while importance-score methods impose heavy client storage and computation overhead.
Approach
Use parameter magnitude as a proxy for importance: each client's submodel is the top-gamma fraction of parameters by absolute value, and a straight-through-style biased gradient lets the mask evolve with parameter values, so no per-parameter importance scores need to be stored.
Key results
- On CIFAR-10, FIARSE achieves 73.75% global accuracy versus 67.37% for ScaleFL and 66.05% for HeteroFL.
- On CIFAR-100, FIARSE reaches 38.63% global accuracy versus 37.56% for ScaleFL and 30.67% for HeteroFL.
- On AGNews, FIARSE reaches 89.61% global accuracy versus 87.19% for FedRolex.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FIARSE instead of ScaleFL | Model-heterogeneous federated learning · CIFAR-10 local test | accuracy: 72.10 -> 77.04 | Helped |
| Use FIARSE instead of ScaleFL | Model-heterogeneous federated learning · CIFAR-10 global test | accuracy: 67.37 -> 73.75 | Helped |
| Use FIARSE instead of ScaleFL | Model-heterogeneous federated learning · CIFAR-100 local test | accuracy: 39.69 -> 41.76 | Helped |
| Use FIARSE instead of ScaleFL | Model-heterogeneous federated learning · CIFAR-100 global test | accuracy: 37.56 -> 38.63 | Helped |
| Use FIARSE instead of ScaleFL | Model-heterogeneous federated learning · AGNews local test | accuracy: 88.02 -> 90.03 | Helped |
| Use FIARSE instead of ScaleFL | Model-heterogeneous federated learning · AGNews global test | accuracy: 87.66 -> 89.61 | Helped |
Limitations
- Relies on unstructured parameter sparsity, which is only conditionally supported by hardware.
- Theoretical analysis assumes a masked-smoothness and masked-reduction bound; the second error term may not vanish in general.
- Evaluated only on ResNet-18 and RoBERTa-base with CIFAR and AGNews.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 AGNews
Benchmarks: CIFAR-10 CIFAR-100 AGNews
Metrics: test accuracy
Baselines: HeteroFL FedRolex ScaleFL
Builds on: HeteroFL FedRolex Flado straight-through estimator
Closest papers by meaning
- Federated Model Heterogeneous Matryoshka Representation Learning (NeurIPS 2024 · similarity 0.88)
- FedQS: Optimizing Gradient and Model Aggregation for Semi-Asynchronous Federated Learning (NeurIPS 2025 · similarity 0.88)
- An Aggregation-Free Federated Learning for Tackling Data Heterogeneity (CVPR 2024 · similarity 0.88)
- Fake It Till Make It: Federated Learning with Consensus-Oriented Generation (ICLR 2024 · similarity 0.88)
- FedGPS: Statistical Rectification Against Data Heterogeneity in Federated Learning (NeurIPS 2025 · similarity 0.88)
- A Swiss Army Knife for Heterogeneous Federated Learning: Flexible Coupling via Trace Norm (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.