Research map / Federated learning and privacy / Personalized and heterogeneous FL
FedCS: Coreset Selection for Federated Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FedCS, a coreset selection method for federated learning that scores samples by Distance Contrast in a pre-trained feature space, using median-aggregated global class centers and a two-stage (double) pruning strategy.
Problem
Federated learning becomes computationally expensive as client datasets grow; existing coreset selection methods either need a server-side dataset or select hard samples, which increases data distribution imbalance across non-IID clients and hurts aggregation at high pruning rates.
Approach
Score each sample by the absolute difference between its distance to its own global class center and to the nearest other class center (Distance Contrast), aggregating local class centers via median on the server, and prune high-DC samples (rare or redundant) with double pruning across large- and small-capacity classes to keep the coreset balanced.
Key results
- On CIFAR-10 (ResNet-18, α=0.1), FedCS reaches 58.88% accuracy with 44% of data kept, while baselines need roughly twice the data for the same accuracy.
- At pf=0.9 on CIFAR-10 (α=0.1), FedCS achieves 56.46% vs 55.63% for whole dataset, outperforming other pruning methods by 0.94% to 5.82%.
- On CIFAR-100 with ViT-B-16, FedCS improves over other methods by 0.75%-2.42% across pruning rates.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FedCS coreset selection instead of training on the whole dataset | ResNet-18 · CIFAR-10, α=0.1, pf=0.9, pl=0.1 | Top-1 accuracy: 55.63±0.68 -> 56.46±0.88 | Helped |
| Use FedCS coreset selection instead of training on the whole dataset | ViT-B-16 · CIFAR-100, α=0.1, pf=0.9, pl=0.1 | Top-1 accuracy: 77.08±0.43 -> 75.25±0.74 | Hurt |
| Use FedCS coreset selection instead of training on the whole dataset | ResNet-50 · TinyImageNet, α=0.1, pf=0.9, pl=0.1 | Top-1 accuracy: 57.61±0.74 -> 56.83±1.18 | Hurt |
| Add class center aggregation (ablation) | FedCS on CIFAR-10, α=0.1 · CIFAR-10, pf=0.7, pl=0.1 | Top-1 accuracy: 56.26±1.11 -> 58.88±0.86 | Helped |
| Add class center aggregation (ablation) | FedCS on CIFAR-10, α=1.0 · CIFAR-10, pf=0.9, pl=0.1 | Top-1 accuracy: 57.51±0.82 -> 58.42±0.76 | Helped |
| Add double pruning (ablation) | FedCS on CIFAR-10, α=0.1 · CIFAR-10, pf=0.7, pl=0.1 | Top-1 accuracy: 55.37±1.30 -> 58.88±0.86 | Helped |
| Add double pruning (ablation) | FedCS on CIFAR-10, α=1.0 · CIFAR-10, pf=0.9, pl=0.1 | Top-1 accuracy: 55.75±0.93 -> 58.42±0.76 | Helped |
Surprising findings
- Pruned coresets (e.g., FedCS at pf=0.9 on CIFAR-10 with α=0.1) can exceed full-dataset training accuracy, which the authors attribute to reduced overfitting from data distribution imbalance.
Limitations
- Relies on a pre-trained network (several pre-training communication rounds) to extract features, and the pruning hyperparameters (beta, pruning ratios) are tuned per setting.
- Convergence analysis relies on strong assumptions (L-smooth, mu-strongly convex, bounded gradients) that may not hold for deep networks.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 CINIC-10 TinyImageNet
Benchmarks: CIFAR-10 CIFAR-100 TinyImageNet
Metrics: Top-1 accuracy
Baselines: EL2N Forgetting Moderate GradND Random GM (Global Moderate)
Builds on: FedAvg Moderate coreset selection EL2N Dirichlet non-IID partitioning
Closest papers by meaning
- Relaxed Contrastive Learning for Federated Learning (CVPR 2024 · similarity 0.87)
- Improved Coresets for Vertical Federated Learning: Regularized Linear and Logistic Regressions (ICML 2025 · similarity 0.86)
- Heterogeneity-Guided Client Sampling: Towards Fast and Efficient Non-IID Federated Learning (NeurIPS 2024 · similarity 0.86)
- Federated Causally Invariant Feature Learning (AAAI 2025 · similarity 0.86)
- FedSC: Provable Federated Self-supervised Learning with Spectral Contrastive Objective over Non-i.i.d. Data (ICML 2024 · similarity 0.85)
- Ranking-based Client Imitation Selection for Efficient Federated Learning (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.