Research map / Federated learning and privacy / Communication-efficient FL
The Promise of Serverless Computing within Peer-to-Peer Architectures for Distributed ML Training
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
A doctoral-consortium style AAAI-24 paper outlining a thesis on combining serverless computing with peer-to-peer distributed ML training, using Redis/RedisAI for in-database gradient averaging and model updates.
Problem
Serverless computing's statelessness, restricted function communication, and limited execution time hinder distributed ML training, and the combination of serverless with peer-to-peer (P2P) architectures is understudied compared with parameter server designs.
Approach
Run per-peer gradient computation in serverless functions, store results in a database, and perform gradient averaging and model updates inside the database (modified RedisAI, Redis scripting) to cut communication overhead, with Byzantine-tolerant aggregation among peers.
Key results
- Serverless gradient computation within peers reported a 97.34% improvement in gradient computation time versus traditional P2P training, at up to 5.4 times higher cost (from prior IC2E 2023 work cited).
- Modified RedisAI supports direct ML model updates inside the database to reduce communication overhead from stateless serverless functions.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use serverless computing for parallel batch processing within each peer | P2P distributed ML training · Distributed ML training | gradient computation time: 97.34% improvement | Helped |
| Use serverless computing for parallel batch processing within each peer | P2P distributed ML training · Distributed ML training | cost: up to 5.4 times more expensive | Hurt |
| Use Redis scripting for gradient averaging | P2P distributed ML training · Distributed ML training | time overhead | Helped |
Surprising findings
- Serverless within peers yields large compute-time gains but up to 5.4x higher cost than traditional P2P training.
Limitations
- Communication overhead from stateless serverless functions requires constant database interaction.
- Thesis is in early stage; comprehensive P2P vs PS comparison is future work.
- Results cited from prior publications rather than presented in detail here.
Experimental setup
Metrics: gradient computation time cost
Baselines: traditional P2P training parameter server serverless training
Builds on: RedisAI Redis parameter server architecture
Closest papers by meaning
- Incorporating Serverless Computing into P2P Networks for ML Training: In-Database Tasks and Their Scalability Implications (Student Abstract) (AAAI 2024 · similarity 0.95)
- Coded Computing for Resilient Distributed Computing: A Learning-Theoretic Framework (NeurIPS 2024 · similarity 0.84)
- Fault Tolerant ML: Efficient Meta-Aggregation and Synchronous Training (ICML 2024 · similarity 0.84)
- Weight for Robustness: A Comprehensive Approach towards Optimal Fault-Tolerant Asynchronous ML (NeurIPS 2024 · similarity 0.83)
- Cheaper and Faster: Distributed Deep Reinforcement Learning with Serverless Computing (AAAI 2024 · similarity 0.82)
- HyperPrism: An Adaptive Non-linear Aggregation Framework for Distributed Machine Learning over Non-IID Data and Time-varying Communication Links (NeurIPS 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.