Research map / Federated learning and privacy / Personalized and heterogeneous FL
Self-Driven Entropy Aggregation for Byzantine-Robust Heterogeneous Federated Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SDEA, a Byzantine-robust aggregation for heterogeneous federated learning that learns client aggregation weights using entropy minimization and diversity maximization on unlabeled random public data, plus cooperative clustering of benign weights.
Problem
Byzantine-robust aggregation in heterogeneous (non-IID) federated learning typically relies on homogeneous client data or a related labeled proxy dataset, which is costly or unavailable in practice.
Approach
Treat client aggregation weights as learnable parameters optimized so the global model gives sharp (low-entropy) per-instance predictions and diverse batch-level predictions on random unrelated public data; then cluster weights with FINCH and equally average the benign cluster.
Key results
- On CIFAR-10 (beta=0.5, Phi=0.2) with random noise attack, SDEA outperforms the best counterpart by 4.49% (69.21% accuracy).
- On MNIST, SDEA with random public data stays around 98-99% accuracy across attacks, while proxy-dataset baselines FLTrust and Sageflow degrade under domain shift (e.g., Sageflow drops from 96.70 with SVHN to 89.51 with SYN under symmetry flipping).
- Ablation on CIFAR-10 shows combining instance sharpness, class diversity, and cooperative cluster gives the best accuracy (e.g., 67.68% PairF, 69.21% RanN).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use SDEA instead of the strongest listed baseline | Cifar-10 federated learning · Cifar-10, β=0.5, Φ=0.2, PairF | accuracy: 67.12 -> 67.68 | Helped |
| use SDEA instead of the strongest listed baseline | Cifar-10 federated learning · Cifar-10, β=0.5, Φ=0.2, SymF | accuracy: 64.72 -> 65.82 | Helped |
| use SDEA instead of the strongest listed baseline | Cifar-10 federated learning · Cifar-10, β=0.5, Φ=0.2, RanN | accuracy: 64.72 -> 69.21 | Helped |
| use SDEA instead of the strongest listed baseline | Cifar-10 federated learning · Cifar-10, β=0.5, Φ=0.2, LIE | accuracy: 64.48 -> 68.32 | Helped |
| use SDEA instead of the strongest listed baseline | Cifar-10 federated learning · Cifar-10, β=0.5, Φ=0.2, MiMa | accuracy: 64.29 -> 67.47 | Helped |
| use SDEA instead of the strongest listed baseline | Cifar-10 federated learning · Cifar-10, β=0.5, Φ=0.2, MiSu | accuracy: 64.02 -> 67.80 | Helped |
| add Instance Sharpness (IS) (ablation) | SDEA with CD and CC · Cifar-10, β=0.5, Φ=0.2, PairF | accuracy: 66.96 -> 67.68 | Helped |
| add Class Diversity (CD) (ablation) | SDEA with IS and CC · Cifar-10, β=0.5, Φ=0.2, PairF | accuracy: 58.75 -> 67.68 | Helped |
| add Cooperative Cluster (CC) (ablation) | SDEA with IS and CD · Cifar-10, β=0.5, Φ=0.2, PairF | accuracy: 51.83 -> 67.68 | Helped |
Surprising findings
- Naively minimizing instance entropy alone collapses performance (e.g., 10% accuracy under some attacks), because learnable weights concentrate on a few clients with sharp predictions.
- Clustering-based averaging with FINCH outperforms K-Means and DBSCAN on CIFAR-10 weight-cluster comparison.
Limitations
- Fails when the public dataset is meaningless (e.g., random noise), as outputs carry no useful information.
- Not designed for backdoor attacks, since malicious backdoored clients can produce confident outputs indistinguishable by entropy.
- Equal weighting among benign clients is probably not globally optimal.
Experimental setup
Datasets: CIFAR-10 MNIST Fashion-MNIST Tiny-ImageNet Market1501 USPS SVHN SYN
Benchmarks: CIFAR-10 MNIST Fashion-MNIST
Metrics: Top-1 accuracy
Baselines: Multi Krum FoolsGold DnC Trim Median Bulyan RFA FLTrust Sageflow
Builds on: FedAvg FedProx FINCH FLTrust Sageflow
Closest papers by meaning
- Efficient Federated Learning against Byzantine Attacks and Data Heterogeneity via Aggregating Normalized Gradients (NeurIPS 2025 · similarity 0.90)
- Rethinking Byzantine Robustness in Federated Recommendation from Sparse Aggregation Perspective (AAAI 2025 · similarity 0.89)
- Noise-Aware Algorithm for Heterogeneous Differentially Private Federated Learning (ICML 2024 · similarity 0.89)
- A Huber Loss Minimization Approach to Byzantine Robust Federated Learning (AAAI 2024 · similarity 0.88)
- FedAWA: Adaptive Optimization of Aggregation Weights in Federated Learning Using Client Vectors (CVPR 2025 · similarity 0.88)
- FedAA: A Reinforcement Learning Perspective on Adaptive Aggregation for Fair and Robust Federated Learning (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.