Research map / Federated learning and privacy / Personalized and heterogeneous FL
pFedES: Generalized Proxy Feature Extractor Sharing for Model Heterogeneous Personalized Federated Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes pFedES, a model-heterogeneous personalized FL method that shares a small proxy homogeneous CNN feature extractor placed before each client's heterogeneous model, trained with alternating freeze-and-train steps.
Problem
Model-heterogeneous personalized federated learning, where clients train personalized heterogeneous local models while avoiding public datasets, high communication/computation costs, and model-structure leakage.
Approach
Insert a small shared homogeneous CNN feature extractor before each client's private heterogeneous model; clients alternately train the heterogeneous model on extractor-enhanced and original data, then train the extractor, and the server aggregates only the extractors (FedAvg-style), so heterogeneous models never leave clients.
Key results
- pFedES achieves up to 1.29% higher average accuracy than the best of 9 baselines across MNIST, CIFAR-10 and CIFAR-100 in model-homogeneous and model-heterogeneous settings.
- Compared with FedGH on CIFAR, pFedES reduces communication cost to 1/224 (99.6% saving) and computation cost to 1/5.85 (82.9% saving) at the target accuracy.
- Theoretical analysis proves a non-convex convergence rate of O(1/T) for a client's heterogeneous local model.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use pFedES instead of FedGH | model-heterogeneous FL · MNIST, N=10, C=100% | test accuracy (%): 99.95±0.26 -> 99.96±0.14 | Helped |
| Use pFedES instead of FedGH | model-heterogeneous FL · CIFAR-10, N=10, C=100% | test accuracy (%): 96.55±0.62 -> 96.70±0.09 | Helped |
| Use pFedES instead of FedGH | model-heterogeneous FL · CIFAR-100, N=10, C=100% | test accuracy (%): 72.60±0.94 -> 73.89±0.26 | Helped |
| Use pFedES instead of FedGH | model-heterogeneous FL · MNIST, N=50, C=20% | test accuracy (%): 99.96±0.28 -> 99.98±0.19 | Helped |
| Use pFedES instead of FedGH | model-heterogeneous FL · CIFAR-10, N=50, C=20% | test accuracy (%): 95.59±0.51 -> 95.79±0.02 | Helped |
| Use pFedES instead of FedGH | model-heterogeneous FL · CIFAR-100, N=50, C=20% | test accuracy (%): 63.29±0.15 -> 64.32±0.11 | Helped |
| Use pFedES instead of FedGH | model-heterogeneous FL · MNIST, N=100, C=10% | test accuracy (%): 99.60±0.05 -> 99.62±0.23 | No clear effect |
| Use pFedES instead of FedGH | model-heterogeneous FL · CIFAR-10, N=100, C=10% | test accuracy (%): 92.51±0.18 -> 92.72±0.07 | Helped |
| Use pFedES instead of FedGH | model-heterogeneous FL · CIFAR-100, N=100, C=10% | test accuracy (%): 53.69±0.30 -> 54.40±0.15 | Helped |
| Share proxy homogeneous feature extractors instead of complete models | pFedES vs FedGH · CIFAR-10 and CIFAR-100, model-heterogeneous FL | communication cost savings: 99.6% | Helped |
| Use pFedES instead of FedGH | model-heterogeneous FL · CIFAR-10 and CIFAR-100 | computation cost savings: 82.9% | Helped |
Surprising findings
- On MNIST, pFedES shows only slight accuracy differences versus the best baselines, which the authors attribute to MNIST being near saturation.
Limitations
- Evaluated only on small CNN models and three image classification benchmarks with supervised learning.
- Feature extractor structure is a small two-layer CNN; other structures are only claimed compatible without testing in the main text.
- Convergence analysis relies on smoothness and bounded-variance assumptions.
Experimental setup
Datasets: MNIST CIFAR-10 CIFAR-100
Benchmarks: MNIST CIFAR-10 CIFAR-100
Metrics: average test accuracy communication cost (transmitted parameters) computation cost (FLOPs)
Baselines: Standalone LG-FedAvg FedGH FML FedKD FedAPEN FD FedProto FedTGP
Builds on: FedAvg FedProto FedGH FML FedRep
Closest papers by meaning
- pFedClub: Controllable Heterogeneous Model Aggregation for Personalized Federated Learning (NeurIPS 2024 · similarity 0.90)
- Personalized Federated Learning via Feature Distribution Adaptation (NeurIPS 2024 · similarity 0.90)
- PFedCS: A Personalized Federated Learning Method for Enhancing Collaboration among Similar Classifiers (AAAI 2025 · similarity 0.88)
- Federated Model Heterogeneous Matryoshka Representation Learning (NeurIPS 2024 · similarity 0.88)
- Tackling Feature-Classifier Mismatch in Federated Learning via Prompt-Driven Feature Transformation (NeurIPS 2025 · similarity 0.88)
- Flexible Sharpness-Aware Personalized Federated Learning (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.