Research map / Federated learning and privacy / Communication-efficient FL
AFL: A Single-Round Analytic Approach for Federated Learning with Pre-trained Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes analytic federated learning (AFL), a gradient-free FL method that uses closed-form least-squares local training on frozen pre-trained features and a single-round absolute aggregation law that is invariant to data partitioning.
Problem
Federated learning with pre-trained models relies on iterative gradient-based local training and multi-round aggregation, which suffers under severe non-IID data, large client counts, slow convergence, and high communication cost.
Approach
Reformulate each client's classifier head as a regularized least-squares problem on frozen backbone embeddings (one epoch), and derive an exact pairwise aggregation law (AA law) that recovers the joint-training solution in a single round; a regularization intermediary (RI) restores optimality when data is rank-deficient with many clients.
Key results
- AFL reaches 80.75% on CIFAR-10, 58.56% on CIFAR-100 and 54.67% on Tiny-ImageNet with identical results across all non-IID settings, versus FedAvg dropping from 56.62% to 32.99% on CIFAR-100 under alpha 0.1 to 0.01.
- AFL is invariant to client number: accuracy stays identical from 100 to 1000 clients while FedAvg falls from 56.57% to 41.01% on CIFAR-100.
- AFL completes training in 236.61s on CIFAR-100 versus roughly 30,000s to 50,000s for multi-round baselines, a reported 150x-200x speedup.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use AFL instead of FedDyn | Federated learning with a frozen ResNet-18 pretrained on ImageNet-1k · CIFAR-100, α = 0.01 | top-1 accuracy (%): 36.12±0.08 -> 58.56±0.00 | Helped |
| Use AFL instead of FedDyn | Federated learning with a frozen ResNet-18 pretrained on ImageNet-1k · Tiny-ImageNet, α = 0.01 | top-1 accuracy (%): 35.19±0.06 -> 54.67±0.00 | Helped |
| Use AFL instead of FedDyn | Federated learning with a frozen ResNet-18 pretrained on ImageNet-1k · CIFAR-100, s = 10 | top-1 accuracy (%): 61.09±0.09 -> 58.56±0.00 | Hurt |
| Add RI restoration to regularized AFL (ablation) | AFL on CIFAR-100 with K=1000 and γ=100 · CIFAR-100, K=1000, γ=100 | top-1 accuracy (%): 49.62 -> 58.56 | Helped |
| Add RI restoration to regularized AFL (ablation) | AFL on CIFAR-100 with K=1000 and γ=10 · CIFAR-100, K=1000, γ=10 | top-1 accuracy (%): 55.77 -> 58.56 | Helped |
| Add RI restoration to regularized AFL (ablation) | AFL on CIFAR-100 with K=500 and γ=100 · CIFAR-100, K=500, γ=100 | top-1 accuracy (%): 51.77 -> 58.56 | Helped |
Surprising findings
- Without the RI process, AFL collapses to 0.75% to 1.11% accuracy at K=500-1000 with gamma=0, and accumulated regularization costs up to 9 points at gamma=100.
- Under extreme non-IID partitions, FedAvg and other baselines degrade sharply while AFL is unchanged, and FedDyn is the only baseline competitive at mild heterogeneity.
Limitations
- Requires a well-trained frozen feature extractor; the method is restricted to linear classifiers and may be less effective on nonlinear distributions.
- Requires all clients to finish local computation before aggregation, so partial participation and stragglers are not handled.
- Evaluated only on image classification benchmarks with frozen ImageNet-pretrained backbones.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 Tiny-ImageNet ImageNet-1k
Benchmarks: CIFAR-10 CIFAR-100 Tiny-ImageNet
Metrics: top-1 accuracy training time accuracy vs number of clients
Baselines: FedAvg FedProx MOON FedGen FedDyn FedNTD FedDisco
Builds on: analytic learning FedAvg pseudoinverse learning MP inverse partition
Closest papers by meaning
- An Aggregation-Free Federated Learning for Tackling Data Heterogeneity (CVPR 2024 · similarity 0.88)
- FedQS: Optimizing Gradient and Model Aggregation for Semi-Asynchronous Federated Learning (NeurIPS 2025 · similarity 0.88)
- Revisiting Ensembling in One-Shot Federated Learning (NeurIPS 2024 · similarity 0.87)
- TurboSVM-FL: Boosting Federated Learning through SVM Aggregation for Lazy Clients (AAAI 2024 · similarity 0.87)
- Momentum-Driven Adaptivity: Towards Tuning-Free Asynchronous Federated Learning (ICML 2025 · similarity 0.87)
- Convergence Analysis of Split Federated Learning on Heterogeneous Data (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.