Research map / Federated learning and privacy / Personalized and heterogeneous FL
Causality Inspired Federated Learning for OOD Generalization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FedUni, a federated learning architecture that extracts the union of all causal features (comprehensive extractor plus inactive-causal-feature compressor) with a counterfactual-generator-based causal intervention module, to improve OOD generalization.
Problem
OOD generalization in federated learning, where global feature extractors trained for inter-client invariance discard client-specific causal features useful for unseen target clients.
Approach
Instead of keeping only causal features invariant across clients, the global extractor captures the union of all causal feature types, and a learned soft feature compressor masks out inactive causal features; a client-side counterfactual generator (with MI-based diversity loss and L1 norm regularization) simulates distribution shifts for contrastive invariance.
Key results
- On Digits, FedUni reaches 66.65% average accuracy (ablation) versus 52.25% for FedAvg; on PACS average is 76.24% vs 65.44% FedAvg per the main table.
- On CMNIST OOD-worst accuracy improves to 56.41% from 27.63% for FedAvg.
- Hyperparameter sensitivity on CMNIST shows maximum fluctuation of 2.57%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FedUni instead of FedSDR | FL OOD generalization · Waterbirds OOD-avg | accuracy (%): 80.60 -> 83.29 | Helped |
| Use FedUni instead of MOON | FL OOD generalization · CMNIST OOD-avg | accuracy (%): 73.92 -> 88.03 | Helped |
| Use FedUni instead of FedAvg | FL OOD generalization · CFMNIST OOD-worst | accuracy (%): 44.52 -> 46.25 | Helped |
| Use FedUni instead of FedSR | FL OOD generalization · Digits Avg. | accuracy (%): 57.24±0.18 -> 66.85±0.64 | Helped |
| Use FedUni instead of FedSDR | FL OOD generalization · PACS Avg. | accuracy (%): 69.36±0.41 -> 76.24±0.60 | Helped |
| Add the causal intervention generator to the FedAvg base model (ablation) | FedAvg on Digits · Digits Avg. | accuracy (%): 52.26 -> 63.37 | Helped |
| Add the inactive causal feature compressor to the FedAvg base model (ablation) | FedAvg on Digits · Digits Avg. | accuracy (%): 52.26 -> 57.00 | Helped |
| Add both the causal intervention generator and inactive causal feature compressor (ablation) | FedAvg on Digits · Digits Avg. | accuracy (%): 52.26 -> 66.65 | Helped |
| Vary α, β, and λ across the reported ranges (ablation) | FedUni on CMNIST · CMNIST | accuracy (%): 2.57% | No clear effect |
Surprising findings
- Simply aggregating the inactive-causal path (ablation without causal intervention) still improves over base, and the compressor alone yields 57.00% on Digits versus 52.26% base.
Limitations
- Relies on a restructured SCM and approximations (parameter-free L1 approximations of entropy) whose tightness is not validated empirically.
- Evaluated only on image classification benchmarks with small backbones (ResNet-18, AlexNet, MLP).
- Hyperparameters alpha, beta, lambda are set per dataset and sensitivity is only shown on CMNIST.
Experimental setup
Datasets: Waterbirds ColoredMNIST ColoredFMNIST Digits (MNIST, MNIST-M, SVHN, SYN, USPS) PACS
Benchmarks: Waterbirds CMNIST CFMNIST Digits PACS
Metrics: test accuracy (ID-avg, ID-worst, OOD-avg, OOD-worst) Fisher score feature redundancy
Baselines: FedAvg FedProx Scaffold MOON FedSR FedIIR FedDG-GA FedSDR
Builds on: FedSR FedIIR Invariant Risk Minimization SimCLR-style contrastive loss FedAvg
Closest papers by meaning
- Causally Motivated Personalized Federated Invariant Learning with Shortcut-Averse Information-Theoretic Regularization (ICML 2024 · similarity 0.89)
- Federated Causally Invariant Feature Learning (AAAI 2025 · similarity 0.88)
- Federated Causal Structure Learning with Non-identical Variable Sets (ICML 2025 · similarity 0.86)
- FOOGD: Federated Collaboration for Both Out-of-distribution Generalization and Detection (NeurIPS 2024 · similarity 0.86)
- On the Out-of-Distribution Generalization of Self-Supervised Learning (ICML 2025 · similarity 0.86)
- Tackling Feature-Classifier Mismatch in Federated Learning via Prompt-Driven Feature Transformation (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.