Research map / Federated learning and privacy / Personalized and heterogeneous FL

Geometric Knowledge-Guided Localized Global Distribution Alignment for Federated Learning

CVPR 2025 · Federated learning and privacy › Personalized and heterogeneous FL · approach: Non-IID federated learning · 8 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes GGEUR, which estimates the global covariance/geometric shape of CLIP embedding distributions from local client statistics and uses it to augment embeddings locally, addressing label and domain skew in federated learning.

Problem

Data heterogeneity (label skew, domain skew, and their coexistence) in federated learning causes misalignment between local and global data distributions, leading to divergent local optimization and unstable global training.

Approach

Approximates the global covariance matrix from local class covariances and means without sharing data, derives the global geometric shape (eigenvectors and eigenvalues) per class, and uses random linear combinations of these eigenvectors to augment local CLIP embeddings; in multi-domain settings, class prototypes from other domains are augmented to simulate cross-domain global distributions.

Key results

• On CIFAR-100 with beta=0.1, FedAvg (CLIP+MLP) improves from 68.22 to 77.70 with GGEUR, outperforming FedMix, FedGen, and FedFA.
• On Digits, GGEUR raises FedAvg (CLIP+MLP) average accuracy from 76.60 to 82.36 and lowers cross-domain STD from 16.25 to 13.84.
• On Office-Home-LDS (beta=0.1), GGEUR improves FedAvg (CLIP+MLP) average accuracy by 13.85 points (70.14 to 83.99) and reduces STD from 8.89 to 5.36.

Measured effects reported in the paper

Surprising findings

• Simply using CLIP features with a single-layer MLP already outperforms many existing FL methods on domain-skew datasets, which the authors attribute to foundation model advances.
• FedNTD without CLIP achieves 56.43 average on Digits but its CLIP+MLP variant reaches 78.86, showing backbone choice dominates some comparisons.

Limitations

• Geometric similarity across domains is shown empirically on Digits only, so cross-domain shared geometry may not generalize.
• Gaussian-style random augmentation along eigen-directions relies on the assumption that the top eigenvector structure captures the global distribution; accuracy of covariance approximation under extreme heterogeneity is not analyzed in the main text.
• Evaluation uses CLIP ViT-B/16 backbones, so gains may depend on the strength of the pretrained foundation model.

Experimental setup

Datasets: CIFAR-10 CIFAR-100 Tiny-ImageNet Digits Office-Caltech PACS Office-Home

Benchmarks: CIFAR-10 CIFAR-100 Tiny-ImageNet Digits Office-Caltech PACS Office-Home-LDS

Metrics: Top-1 accuracy standard deviation of accuracy across domains

Baselines: FedMix FEDGEN FedFA FedTPG FedCLIP SCAFFOLD MOON FedDyn FedOPT FedProto FedNTD

Builds on: FedAvg FedFA FedMix FedGen CLIP

Closest papers by meaning

• FedScar: Correcting Geometric Bias for Flatness-Consistent Federated Learning (ICML 2026 · similarity 0.89)
• FedPDG: Prediction Discrepancy–Guided Data Generation for Heterogeneous Federated Learning (ICML 2026 · similarity 0.88)
• Federated Graph Learning under Domain Shift with Generalizable Prototypes (AAAI 2024 · similarity 0.86)
• FLUX: Efficient Descriptor-Driven Clustered Federated Learning under Arbitrary Distribution Shifts (NeurIPS 2025 · similarity 0.86)
• FedGPS: Statistical Rectification Against Data Heterogeneity in Federated Learning (NeurIPS 2025 · similarity 0.86)
• One Arrow, Two Hawks: Sharpness-aware Minimization for Federated Learning via Global Model Trajectory (ICML 2025 · similarity 0.86)