Research map / Federated learning and privacy / Personalized and heterogeneous FL
Diffusion-Based Active Learning for Distributed Client Manifolds
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a distributed active learning method that selects clients and local points by diffusing predicted entropy reductions over a graph of client pseudo-parameter embeddings, without accessing local data.
Problem
Active learning in privacy-preserving distributed settings where clients hold unshared local data and the server must decide which clients to label, accounting for inter-client relationships.
Approach
Embed clients as normalized pseudo-parameter updates (from pseudo-labels on unlabeled data) forming a client manifold graph, then score clients by simulating anisotropic diffusion of average entropy so that labeling a client's impact on all clients' uncertainty is captured, promoting both exploitation and diversity.
Key results
- Ours was statistically significantly better than random selection on 93.6% of dataset/label-count cases and not significantly worse on any.
- Uncertainty-based client sampling (Ent+Ent, Ent+F-AL) beat random on only 28.2% of cases and was worse on 17.3%.
- Experiments used non-IID Dirichlet (alpha in [0.1,0.2]) and Shard allocations across 100 or 300 clients, averaged over ten random allocations.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use diffusion-based global client-impact selection instead of random client selection | Ours · All five datasets and both data allocations | accuracy: 93.6% | Helped |
| use diffusion-based global client-impact selection instead of random client selection | Ours · All five datasets and both data allocations | accuracy | No clear effect |
| select clients by average entropy rather than random selection | Ent+Ent and Ent+F-AL · CINIC-10, Fashion-MNIST, CIFAR-10, and CIFAR-100 | accuracy | Mixed |
| use hybrid F-AL point selection instead of pure uncertainty sampling | Distributed AL · Individual clients across the evaluated datasets | accuracy | Helped |
| disable anisotropicity by setting ρ=0 (ablation) | Ours · CIFAR-10 | average accuracy: -0.27% | Hurt |
Surprising findings
- Naive uncertainty-based client selection often underperforms random client selection, sometimes selecting clients with limited global contribution.
Limitations
- Hyperparameters (K, H, delta, sigma values) were not optimized and their tuning is difficult given few labels.
- Requires pseudo-label-based parameter embeddings, which may be unreliable in early AL rounds.
- Experiments are limited to image classification with a single pre-trained backbone.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 Fashion-MNIST EMNIST letters CINIC-10
Benchmarks: CIFAR-10 CIFAR-100 Fashion-MNIST EMNIST letters CINIC-10
Metrics: accuracy improvement over random selection statistical significance (Student's t-test, alpha 0.95)
Baselines: Rand Ent+Ent (Ahmed et al. 2020) Rand+Ent Ent+F-AL (Ahn et al. 2022) Rand+F-AL
Builds on: FedAvg (McMahan et al. 2017) Anisotropic diffusion on graphs (Szlam et al. 2008; Kim et al. 2015) F-AL (Ahn et al. 2022) Federated AL (Ahmed et al. 2020)
Closest papers by meaning
- SP-CACW: Convergence-Aware Client Weighting for Selfish Personalized Learning (NeurIPS 2026 · similarity 0.84)
- Learning from End User Data with Shuffled Differential Privacy over Kernel Densities (ICLR 2025 · similarity 0.83)
- Beyond the Federation: Topology-aware Federated Learning for Generalization to Unseen Clients (ICML 2024 · similarity 0.83)
- A Bayesian Approach for Personalized Federated Learning in Heterogeneous Settings (NeurIPS 2024 · similarity 0.83)
- FLUX: Efficient Descriptor-Driven Clustered Federated Learning under Arbitrary Distribution Shifts (NeurIPS 2025 · similarity 0.83)
- Little Is Enough: Boosting Privacy by Sharing Only Hard Labels in Federated Semi-Supervised Learning (AAAI 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.