Research map / Federated learning and privacy / Personalized and heterogeneous FL
Causally Motivated Personalized Federated Invariant Learning with Shortcut-Averse Information-Theoretic Regularization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FedPIN, a causally motivated personalized federated invariant learning method that uses an information-theoretic shortcut-averse regularizer anchored on a global invariant feature extractor to learn personalized invariant features under distribution shift.
Problem
Personalized federated learning must preserve client-specific features while removing spurious correlations, but personalized features are entangled with shortcuts, so OOD generalization across client test distributions suffers.
Approach
Derives a causal signature from a heterogeneous structured causal model (personalized invariant features are conditionally dependent on global invariant features given Y, while shortcuts are not) and quantifies it as a conditional mutual information term minus an entropy term, approximated via contrastive loss and variance, with a global invariant extractor as anchor.
Key results
- FedPIN reaches 59.8% worst-case accuracy on CFMNIST with 8 clients versus 56.9 for FedSDR, and 73.8% worst-case on WaterBird with 8 clients.
- FedPIN achieves the highest average accuracy on CFMNIST, WaterBird and PACS among compared methods (e.g., 63.1 on CFMNIST 8 clients, 75.8 on WaterBird 8 clients).
- Ablation shows removing the conditional mutual information term (lambda=0) drops CMNIST worst-case accuracy from 53.6 to 20.2.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the FedPIN shortcut-averse invariant-learning method | personalized federated learning · CFMNIST, 8 and 80 clients | worst-case accuracy: 56.9 -> 59.8; 52.8 -> 56.4 | Helped |
| add the FedPIN shortcut-averse invariant-learning method | personalized federated learning · WaterBird, 8 and 80 clients | worst-case accuracy: 65.3 -> 73.8; 60.0 -> 67.9 | Helped |
| add the FedPIN shortcut-averse invariant-learning method | personalized federated learning · PACS, 6 and 60 clients | worst-case accuracy: 52.1 -> 55.4; 48.1 -> 52.3 | Helped |
| remove the conditional mutual information term (λ = 0) (ablation) | FedPIN · CMNIST, CFMNIST, WaterBird, and PACS | worst-case accuracy: 53.6 -> 20.2; 59.8 -> 18.2; 73.8 -> 64.0; 55.4 -> 46.1 | Hurt |
| remove the conditional mutual information term (λ = 0) (ablation) | FedPIN · CMNIST, CFMNIST, WaterBird, and PACS | average accuracy: 55.4 -> 54.5; 63.1 -> 55.7; 75.8 -> 72.8; 58.6 -> 53.0 | Hurt |
| remove the entropy term (γ = 0) (ablation) | FedPIN · CMNIST, CFMNIST, WaterBird, and PACS | worst-case accuracy: 53.6 -> 52.8; 59.8 -> 58.6; 73.8 -> 69.8; 55.4 -> 52.9 | Hurt |
| remove the entropy term (γ = 0) (ablation) | FedPIN · CMNIST, CFMNIST, WaterBird, and PACS | average accuracy: 55.4 -> 55.2; 63.1 -> 63.2; 75.8 -> 75.4; 58.6 -> 56.3 | Hurt |
| increase the number of local epochs R from 5 to 20 (ablation) | FedPIN · WaterBird | worst-case accuracy: 71.8 -> 73.8 -> 73.7 -> 73.3 | Mixed |
| increase the number of local epochs R from 5 to 20 (ablation) | FedPIN · WaterBird | average accuracy: 74.4 -> 75.8 -> 76.2 -> 76.4 | Helped |
| use local fine-tuning (GM-FT) instead of the proposed personalized invariant objective (ablation) | FedPIN global invariant model · CMNIST, CFMNIST, WaterBird, and PACS | worst-case accuracy: 47.5 -> 15.1; 48.2 -> 10.7; 63.4 -> 62.8; 47.2 -> 45.5 | Hurt |
| use L2-norm regularization (GM-L2) instead of the proposed personalized invariant objective (ablation) | FedPIN global invariant model · CMNIST, CFMNIST, WaterBird, and PACS | worst-case accuracy: 47.5 -> 46.9; 48.2 -> 48.6; 63.4 -> 64.9; 47.2 -> 48.2 | Mixed |
Surprising findings
- Standard local fine-tuning and L2 regularization on the global invariant model reduce OOD accuracy (e.g., GM-FT worst-case on CMNIST 15.1 vs GM 47.5), indicating personalization entangles with shortcuts.
Limitations
- Theoretical guarantees rely on assumed SCM and Markov property; results depend on hyperparameters lambda, gamma chosen appropriately.
- Experiments are on small-scale image benchmarks with simulated clients and only ResNet-18 or small MLP backbones.
Experimental setup
Datasets: CMNIST CFMNIST WaterBird PACS
Benchmarks: CMNIST CFMNIST WaterBird PACS
Metrics: worst-case test accuracy average test accuracy
Baselines: FedAvg DRFA FedSR FedIIR pFedMe Ditto FTFA FedRep FedRoD FedPAC FedSDR
Builds on: IRM FedIIR FedSDR Ditto FedRep
Closest papers by meaning
- Causality Inspired Federated Learning for OOD Generalization (ICML 2025 · similarity 0.89)
- Flexible Sharpness-Aware Personalized Federated Learning (AAAI 2025 · similarity 0.87)
- Adaptive Latent-Space Constraints in Personalized Federated Learning (NeurIPS 2025 · similarity 0.87)
- Federated Causally Invariant Feature Learning (AAAI 2025 · similarity 0.87)
- Tackling Feature-Classifier Mismatch in Federated Learning via Prompt-Driven Feature Transformation (NeurIPS 2025 · similarity 0.87)
- Understanding the Statistical Accuracy-Communication Trade-off in Personalized Federated Learning with Minimax Guarantees (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.