Research map / Federated learning and privacy / Personalized and heterogeneous FL
FedCFA: Alleviating Simpson’s Paradox in Model Aggregation with Counterfactual Federated Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FedCFA, a federated learning framework that uses counterfactual transformations of local features (replaced with global average data features) plus a factor decorrelation loss to mitigate Simpson's Paradox in model aggregation.
Problem
Federated learning under data imbalance and heterogeneity, where the aggregated global model can mis-capture the global data distribution because local data trends reverse or vanish relative to the global data (Simpson's Paradox).
Approach
Generate counterfactual positive and negative samples by swapping gradient-selected critical/dispensable latent factors with factors from a shared global average dataset, use contrastive-style classification on them, and decorrelate factors with a Pearson-based FDC loss so that local distributions move closer to the global one.
Key results
- On CIFAR100 under Dir100(0.6), FedCFA reaches 49.18% vs 41.43% for the best baseline FedMix (about 7.75 points above the top baseline as stated in text).
- On CIFAR10 Dir60(0.2), FedCFA reaches the 75.5% target in 375 rounds vs. FedMix's 610 rounds; other baselines fail within 1000 rounds in that setting.
- On Tiny-ImageNet Dir60(0.2), FedCFA attains 30.70% vs 28.01% for FedMix.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace FedMix with FedCFA | federated learning · CIFAR100 Dir60(0.2), Dir60(0.6), and IID60 | top-1 accuracy (%): 42.51$\pm$0.28 -> 46.96$\pm$1.04; 44.16$\pm$0.26 -> 49.32$\pm$0.20; 45.91$\pm$0.05 -> 48.31$\pm$0.53 | Helped |
| replace FedMix with FedCFA | federated learning · CIFAR10 Dir60(0.2), Dir60(0.6), and IID60 | top-1 accuracy (%): 74.61$\pm$0.74 -> 75.89$\pm$1.00; 78.64$\pm$0.53 -> 82.43$\pm$0.08; 77.90$\pm$0.17 -> 83.36$\pm$0.51 | Helped |
| replace FedMix with FedCFA | federated learning · Tiny-ImageNet Dir60(0.2) and Dir60(0.6) | top-1 accuracy (%): 28.01$\pm$0.19 -> 30.70$\pm$0.68; 32.43$\pm$0.13 -> 32.86$\pm$0.77 | Helped |
| replace FedMix with FedCFA | federated learning · FEMNIST Non-IID | top-1 accuracy (%): 82.31$\pm$0.21 -> 83.19$\pm$0.54 | Helped |
| replace FedAvg with FedCFA | federated learning · Sent140 Non-IID | top-1 accuracy (%): 68.10$\pm$0.48 -> 69.26$\pm$0.37 | Helped |
| add both positive and negative counterfactual modules (ablation) | FedCFA · CIFAR10 Dir60(0.6) | accuracy: 78.17 -> 80.01 | Helped |
| use only the positive counterfactual module instead of both (ablation) | FedCFA · CIFAR10 Dir60(0.6) | accuracy: 80.01 -> 78.86 | Hurt |
| use only the negative counterfactual module instead of both (ablation) | FedCFA · CIFAR10 Dir60(0.6) | accuracy: 80.01 -> 78.87 | Hurt |
| apply counterfactuals at an intermediate layer (Hook 2) rather than Hook 0 or Hook 1 (ablation) | FedCFA ResNet18 · CIFAR10 Dir60(0.6) | top-1 accuracy (%): 77.65 -> 80.01; 77.87 -> 80.01 | Helped |
| apply counterfactuals at Hook 4 instead of Hook 2 (ablation) | FedCFA ResNet18 · CIFAR10 Dir60(0.6) | top-1 accuracy (%): 80.01 -> 75.31 | Hurt |
| apply counterfactuals at an intermediate layer (Hook 2) rather than Hook 0, Hook 1, or Hook 4 (ablation) | FedCFA ResNet18 · CIFAR10 Dir100(0.6) | top-1 accuracy (%): 75.89 -> 80.03; 75.96 -> 80.03; 73.54 -> 80.03 | Helped |
| increase Topk from 8 to 24 (ablation) | FedCFA · CIFAR10 Dir60(0.6) | top-1 accuracy (%): 78.53 -> 80.01 | Helped |
Surprising findings
- Naive augmentations (color jittering, random flipping) do not mitigate the Simpson's Paradox effect in t-SNE analysis.
- On Sent140 and FEMNIST, all methods perform similarly and FedCFA's advantage is slight.
Limitations
- Relies on a global average dataset built by aggregating client means, which raises privacy and communication concerns not analyzed in depth.
- Pearson correlation only captures linear dependence between factors; the authors note non-linear correlation is not handled.
- Hyperparameters (topk, hook layer, lambda values) are tuned on the evaluated settings, with sensitivity showing performance drops outside narrow ranges.
Experimental setup
Datasets: CIFAR10 CIFAR100 Tiny-ImageNet FEMNIST Sent140 MNIST Fashion-MNIST
Benchmarks: CIFAR10 CIFAR100 Tiny-ImageNet FEMNIST Sent140 Fashion-MNIST MNIST with Simpson's Paradox
Metrics: top-1 accuracy communication rounds to target accuracy
Baselines: FedAvg FedProx SCAFFOLD FedPVR q-FedAvg FedMix
Builds on: FedAvg FedProx SCAFFOLD FedMix
Closest papers by meaning
- An Aggregation-Free Federated Learning for Tackling Data Heterogeneity (CVPR 2024 · similarity 0.87)
- FedAWA: Adaptive Optimization of Aggregation Weights in Federated Learning Using Client Vectors (CVPR 2025 · similarity 0.86)
- Bridging Model Heterogeneity in Federated Learning via Uncertainty-based Asymmetrical Reciprocity Learning (ICML 2024 · similarity 0.86)
- FedAA: A Reinforcement Learning Perspective on Adaptive Aggregation for Fair and Robust Federated Learning (AAAI 2025 · similarity 0.86)
- FedFACT: A Provable Framework for Controllable Group-Fairness Calibration in Federated Learning (NeurIPS 2025 · similarity 0.85)
- FedLMT: Tackling System Heterogeneity of Federated Learning via Low-Rank Model Training with Theoretical Guarantees (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.