Research map / Federated learning and privacy / Personalized and heterogeneous FL

Federated Learning for Feature Generalization with Convex Constraints

ICML 2025 · Federated learning and privacy › Personalized and heterogeneous FL · approach: Non-IID federated learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes FedCONST, a federated learning method that adds centralization and orthogonal (convex) constraints to client updates, guided by global weight magnitudes as a proxy for feature strength, to improve generalization across heterogeneous clients.

Problem

Federated learning generalizes poorly under heterogeneous (non-IID) client data: local models overfit their local distributions, and transferable features get distorted during aggregation.

Approach

Constrain local updates with linear convex constraints derived from the global model's weights (centering of updates and projection orthogonal to the global weight vector), so well-learned weights change less and under-learned weights get more updates; the constraints are shared across clients so they survive aggregation and need no extra communication.

Key results

• On CIFAR-10 with LeNet-5 (cross-device, alpha=0.5), FedAvg rises from 46.12% to 54.28% with CONST.
• On CIFAR-10 with ResNet-18 (cross-device, alpha=0.5), FedAvg rises from 54.07% to 66.51% with CONST.
• FedSAM + CONST reaches 39.61% on CIFAR-100 (ResNet-18, cross-silo, alpha=0.5), the best reported in that column.

Measured effects reported in the paper

Surprising findings

• Applying constraints increased the maximum Hessian eigenvalue, which the authors say challenges the usual sharpness-generalization interpretation, so they argue convexity is a better indicator in FL.
• Some baselines lose accuracy under the constraint in certain settings, e.g. MOON + CONST is lower than MOON on one CIFAR-10 cross-silo setting only if read carelessly; in table, FedProx + CONST and others gain in most settings but the gains are not uniform across all columns.

Limitations

• Experiments are limited to CIFAR-10/100 with LeNet-5 and ResNet-18; no larger models or datasets are reported.
• The weight-magnitude-as-GSNR-proxy link is argued heuristically and relies on a stable-state assumption.
• The paper's theoretical claims rest on Gaussian assumptions and a spherical-noise model that are only loosely justified.

Experimental setup

Datasets: CIFAR-10 CIFAR-100

Benchmarks: CIFAR-10 CIFAR-100

Metrics: top-1 test accuracy Hessian eigenvalue ratio (convexity) Hessian trace gradient signal-to-noise ratio client consistency drift diversity

Baselines: FedAvg FedProx MOON SCAFFOLD FedDyn FedSAM

Builds on: FedAvg FedProx MOON SCAFFOLD FedDyn FedSAM GSNR-based dropout for domain generalization

Closest papers by meaning

• FedImpro: Measuring and Improving Client Update in Federated Learning (ICLR 2024 · similarity 0.87)
• FedLoGe: Joint Local and Generic Federated Learning under Long-tailed Data (ICLR 2024 · similarity 0.87)
• Improving Generalization in Federated Learning with Model-Data Mutual Information Regularization: A Posterior Inference Approach (NeurIPS 2024 · similarity 0.87)
• Locally Estimated Global Perturbations are Better than Local Perturbations for Federated Sharpness-aware Minimization (ICML 2024 · similarity 0.87)
• Subspace Constraint and Contribution Estimation for Heterogeneous Federated Learning (CVPR 2025 · similarity 0.86)
• Lessons from Generalization Error Analysis of Federated Learning: You May Communicate Less Often! (ICML 2024 · similarity 0.86)