Research map / Federated learning and privacy / Personalized and heterogeneous FL

Efficient Heterogeneity-Aware Federated Active Data Selection

ICML 2025 · Federated learning and privacy › Personalized and heterogeneous FL · approach: Non-IID federated learning

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes FALE, a one-pass federated active data selection method for non-i.i.d. regression that uses FedSVD-based global leverage score sampling and secure masked global regression, with query-complexity guarantees.

Problem

Federated active learning for regression on non-i.i.d. client data needs label-efficient global data selection without sharing raw data or cross-client query information; existing FAL methods select locally and iteratively.

Approach

Use FedSVD to securely obtain global left-singular-vector leverage scores, sample instances globally by leverage score in a single pass, reweight them, and learn the global regression model on the server from masked features and labels.

Key results

• FALE consistently outperforms baselines across nine tabular regression benchmarks; in paired t-tests vs. the rest of methods it has 675 wins, 41 ties, 4 losses (over 90 comparisons per baseline setting).
• Theory: FALE with O(d log d) queries achieves a constant-factor approximation with probability at least 0.99; with Ω(d log d + d/ε) queries it achieves (1+ε) relative error with probability at least 1/3.
• On image regression (CelebA, IMDB-WIKI) with one-pass budgets of 7,500 and 15,000 queries, FALE significantly outperforms iterative selection methods.

Measured effects reported in the paper

Surprising findings

• Local-only baselines such as LOGO and ACS-FW show increasing test error as more data is queried on some datasets, attributed to client distribution discrepancies.

Limitations

• Theoretical guarantees hold for linear regression models; the experiments use a single-layer model.
• Constant-factor and relative-error guarantees are probabilistic (0.99 or 1/3 success probability).
• Experiments simulate non-i.i.d. splits via Dirichlet binning on regression targets across 10 clients.

Experimental setup

Datasets: ct kegg undir online video wec sydney sarcos diamonds stock protein mlr knn rng CelebA IMDB-WIKI

Benchmarks: ct kegg undir online video wec sydney sarcos diamonds stock protein mlr knn rng CelebA IMDB-WIKI

Metrics: mean squared error paired t-test win/tie/loss

Baselines: Random BAIT ACS-FW CoreSet BADGE LCMD LOGO FALE-local

Builds on: FedSVD leverage score sampling FedAvg LOGO

Closest papers by meaning

• Ranking-based Client Imitation Selection for Efficient Federated Learning (ICML 2024 · similarity 0.87)
• Heterogeneity-Guided Client Sampling: Towards Fast and Efficient Non-IID Federated Learning (NeurIPS 2024 · similarity 0.86)
• (FL)$^2$: Overcoming Few Labels in Federated Semi-Supervised Learning (NeurIPS 2024 · similarity 0.85)
• FADAS: Towards Federated Adaptive Asynchronous Optimization (ICML 2024 · similarity 0.85)
• FedLMT: Tackling System Heterogeneity of Federated Learning via Low-Rank Model Training with Theoretical Guarantees (ICML 2024 · similarity 0.85)
• Reinforcement Active Client Selection for Federated Heterogeneous Graph Learning (AAAI 2025 · similarity 0.84)