Research map / Federated learning and privacy / Communication-efficient FL

Principled Federated Random Forests for Heterogeneous Data

NeurIPS 2026 · Federated learning and privacy › Communication-efficient FL · approach: Efficient local training in FL · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes FedForest, a federated random forest for horizontally partitioned data that reproduces centralized CART splits from aggregated client sufficient statistics, adds quantile-based candidate generation, and allows splits on client indicators.

Problem

Random forests are powerful for tabular data but cannot be trained with gradients in federated settings; existing federated RF heuristics (local ensembling, histogram aggregation) fail to optimize the global impurity criterion under client heterogeneity such as covariate or outcome shift.

Approach

Federate CART split selection by sketching pooled feature CDFs from client quantiles to generate candidates, and evaluate impurity gains exactly via additive sufficient statistics summed across clients; client indicators are treated as a categorical feature so splits on H cost no extra communication.

Key results

• On the covariate-shift synthetic setting, FedForest-AvgImp fails while FedForest-Quantiles and Federated Histogram stay close to centralized RF; local ensembling reaches about 141.2 MSE.
• On the public-dataset benchmark, FedForest Quantiles is typically a top performer; e.g., on Wine (alpha=10) it reaches 0.990 balanced accuracy versus 0.98 for centralized scikit RF.
• On Banknote Authentication across K=1 to 100 clients, FedForest variants stay near centralized RF (about 0.99), while Local Learning drops to 0.55 at K=100.

Surprising findings

• Averaging client-wise impurity gains (AvgImp) fails under covariate shift even though it works under homogeneity, because each client observes only one side of the partition.
• Under the reported benchmarks, the federated neural network (2-layer NN) sometimes outperforms tree methods, e.g., on kin8nm and puma8NH, despite the general tree advantage claimed.

Limitations

• Exact centralized equivalence relies on sending per-candidate sufficient statistics; privacy of aggregated summaries is not formally guaranteed (no differential privacy).
• AvgImp approximation guarantee only holds under homogeneous clients and becomes uninformative when K is comparable to n.
• Quantile candidates give only approximate split fidelity (bounded disagreement 3/(2B)), not exact CART splits.

Experimental setup

Datasets: FedHeart Disease (FLamby) Banknote Authentication diabetes bodyfat space-ga puma8NH cpu-act kin8nm breast-cancer Spectf Bioresponse madelon

Benchmarks: FedHeart Disease (FLamby) Banknote Authentication (K sweep) 13 OpenML/scikit-learn datasets with Dirichlet partitioning synthetic homogeneous/covariate/outcome shift settings

Metrics: test MSE R^2 balanced accuracy

Baselines: Centralized scikit-learn RF Federated Histogram Local Ensembling (Hauschild et al.) Local Learning FedAvg LR FedAvg NN Centralized 2-layer NN

Builds on: CART Random Forests Federated Averaging Federated Histogram (Kalloori et al.)

Closest papers by meaning

• FedRSPO+: A Heterogeneity-aware Algorithm for Decision-focused Federated Learning (NeurIPS 2026 · similarity 0.87)
• Decentralized Sporadic Federated Learning: A Unified Algorithmic Framework with Convergence Guarantees (ICLR 2025 · similarity 0.87)
• Effective and Efficient Federated Tree Learning on Hybrid Data (ICLR 2024 · similarity 0.87)
• Fed-QSSL: A Framework for Personalized Federated Learning under Bitwidth and Data Heterogeneity (AAAI 2024 · similarity 0.86)
• Uncertainty-Based Extensible Codebook for Discrete Federated Learning in Heterogeneous Data Silos (ICML 2025 · similarity 0.86)
• Federated Learning under Periodic Client Participation and Heterogeneous Data: A New Communication-Efficient Algorithm and Analysis (NeurIPS 2024 · similarity 0.86)