Research map / Federated learning and privacy / Personalized and heterogeneous FL
Towards Federated Foundation Models: Scalable Dataset Pipelines for Group-Structured Learning
Zachary Charles, Nicole Elyse Mitchell, Krishna Pillutla, Michael Reneer, Zachary Garrett
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace in-memory format with streaming format for large datasets (ablation) | Dataset Grouper · FedBookCO | time to iterate (s): Out of memory -> 192 | Improved |
| Replace hierarchical format with streaming format (ablation) | Dataset Grouper · FedCCnews | time to iterate (s): >7200 -> 248 | Improved |
| Use in-memory format instead of streaming on small data (ablation) | Dataset Grouper · CIFAR-100 | time to iterate (s): 9.88 -> 0.0783 | Improved |
| Increase cohort size from 8 to 32 | FedAvg with Dataset Grouper · FedC4 | data iteration share of round time (%): 7.78 -> 9.33 | Mixed |
| Apply learning rate schedules (warmup with decay) at the server (ablation) | FedSGD · FedC4 | training loss | Improved |
| Apply learning rate schedules at the server (ablation) | FedAvg · FedC4 | training loss | No clear effect |
| Use FedSGD instead of FedAvg | server training · FedC4 validation | pre-personalization median loss: 5.64 -> 4.93 | Improved |
| Use FedAvg instead of FedSGD | server training · FedC4 validation | post-personalization median loss: 3.38 -> 0.012 | Improved |
| Use FedAvg instead of FedSGD | server training · FedBookCO | post-personalization loss: 4.0 -> 2.9 | Improved |
| Use FedAvg instead of FedSGD | server training · FedBookCO | pre-personalization loss: 4.3 -> 5.0 | Hurt |
| Scale the model to 1B parameters with 4 batches per client (ablation) | FedAvg and FedSGD · FedC4 validation | pre-personalization loss | Improved |
Closest papers by meaning
- FedLLM-Bench: Realistic Benchmarks for Federated Learning of Large Language Models (NeurIPS 2024 · similarity 0.89)
- FlowerTune: A Cross-Domain Benchmark for Federated Fine-Tuning of Large Language Models (NeurIPS 2025 · similarity 0.87)
- FedQS: Optimizing Gradient and Model Aggregation for Semi-Asynchronous Federated Learning (NeurIPS 2025 · similarity 0.86)
- Benchmarking Algorithms for Federated Domain Generalization (ICLR 2024 · similarity 0.86)
- Splitting with Importance-aware Updating for Heterogeneous Federated Learning with Large Language Models (ICML 2025 · similarity 0.86)
- Dual-Personalizing Adapter for Federated Foundation Models (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
