Research map / Federated learning and privacy / Personalized and heterogeneous FL
Knowledge-Aware Parameter Coaching for Personalized Federated Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a knowledge-aware parameter coaching method for personalized federated learning that uses a layer-wise relation cube to weight per-client server regularizers, with convergence proofs for convex and non-convex settings.
Problem
Personalized federated learning under non-IID client data struggles to exploit fine-grained, layer-wise knowledge from other clients efficiently; whole-model aggregation transfers conflicting knowledge and hypernetwork-based layer-wise methods converge slowly.
Approach
A relation cube of layer-wise similarity coefficients between all client pairs builds a personalized server model per client as a linear combination of other clients' layer parameters, which acts as a regularizer for local training; the cube and client parameters are updated alternately via block coordinate descent.
Key results
- On CIFAR10 with 100 clients and alpha=0.1, the method reaches 80.65% accuracy vs 78.58% for FedProto and 78.70% for FedBN (client selection scenario).
- Without client selection (10 clients), the method achieves 97.97% on MNIST (alpha=0.1), the best among compared methods in Table 1.
- Convergence speed on CIFAR10/CIFAR100 is roughly comparable to FedProx and MOON and much faster than layer-wise methods FedBABU, FedBN and pFedLA.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use knowledge-aware parameter coaching instead of FedProx | Personalized federated learning · MNIST, 10 clients, α=0.1 | accuracy (%): 97.93(0.66) -> 97.97(0.75) | Helped |
| Use knowledge-aware parameter coaching instead of MOON | Personalized federated learning · MNIST, 10 clients, α=0.3 | accuracy (%): 97.43(0.66) -> 97.66(0.38) | Helped |
| Use knowledge-aware parameter coaching instead of MOON | Personalized federated learning · FMNIST, 10 clients, α=0.1 | accuracy (%): 96.61(0.35) -> 96.71(0.17) | Helped |
| Use knowledge-aware parameter coaching instead of FedProx | Personalized federated learning · FMNIST, 10 clients, α=0.3 | accuracy (%): 87.81(0.45) -> 88.04(0.46) | Helped |
| Use knowledge-aware parameter coaching instead of MOON | Personalized federated learning · CIFAR10, 10 clients, α=0.1 | accuracy (%): 88.27(0.24) -> 88.32(0.65) | Helped |
| Use knowledge-aware parameter coaching instead of FedBABU | Personalized federated learning · CIFAR10, 10 clients, α=0.3 | accuracy (%): 65.19(0.21) -> 65.75(0.84) | Helped |
| Use knowledge-aware parameter coaching instead of FedBABU | Personalized federated learning · CIFAR100, 10 clients, α=0.1 | accuracy (%): 38.04(0.60) -> 38.25(0.33) | Helped |
| Use knowledge-aware parameter coaching instead of FedProto | Personalized federated learning · CIFAR100, 10 clients, α=0.3 | accuracy (%): 26.99(0.30) -> 27.02(0.58) | Helped |
| Use knowledge-aware parameter coaching instead of FedProx | Personalized federated learning · MNIST, 100 clients, 10% participation, α=0.1 | accuracy (%): 97.41(0.30) -> 97.53(0.61) | Helped |
| Use knowledge-aware parameter coaching instead of FedBN | Personalized federated learning · MNIST, 100 clients, 10% participation, α=0.3 | accuracy (%): 95.56(0.31) -> 96.17(0.21) | Helped |
| Use knowledge-aware parameter coaching instead of FedProto | Personalized federated learning · FMNIST, 100 clients, 10% participation, α=0.1 | accuracy (%): 94.42(0.26) -> 94.57(0.25) | Helped |
| Use knowledge-aware parameter coaching instead of FedProto | Personalized federated learning · FMNIST, 100 clients, 10% participation, α=0.3 | accuracy (%): 90.24(0.15) -> 90.34(0.67) | Helped |
Surprising findings
- pFedLA performs far worse than most baselines on several datasets (e.g., 58.35% on MNIST with 100 clients, alpha=0.1), which the authors attribute to hard-to-train hypernetworks.
Limitations
- Convergence speed is only analyzed empirically on CIFAR10/CIFAR100; theory gives no explicit rate for the non-convex case.
- Experiments are limited to image classification with small CNN and ResNet18 models on four standard datasets.
- Hyperparameters lambda and beta need dataset/heterogeneity-specific tuning, as shown in the sensitivity analysis.
Experimental setup
Datasets: MNIST FMNIST CIFAR10 CIFAR100
Benchmarks: MNIST FMNIST CIFAR10 CIFAR100
Metrics: mean test accuracy convergence speed (accuracy vs communication round)
Baselines: local training FedAvg FedProx FedAMP MOON FedProto FedBABU FedBN pFedLA
Builds on: FedAMP FedProto pFedLA FedAvg FedProx
Closest papers by meaning
- FedAS: Bridging Inconsistency in Personalized Federated Learning (CVPR 2024 · similarity 0.88)
- Understanding the Statistical Accuracy-Communication Trade-off in Personalized Federated Learning with Minimax Guarantees (ICML 2025 · similarity 0.88)
- PerAda: Parameter-Efficient Federated Learning Personalization with Generalization Guarantees (CVPR 2024 · similarity 0.88)
- FedSelect: Personalized Federated Learning with Customized Selection of Parameters for Fine-Tuning (CVPR 2024 · similarity 0.88)
- PeFLL: Personalized Federated Learning by Learning to Learn (ICLR 2024 · similarity 0.87)
- Flexible Sharpness-Aware Personalized Federated Learning (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.