Research map / Federated learning and privacy / Communication-efficient FL
HyperPrism: An Adaptive Non-linear Aggregation Framework for Distributed Machine Learning over Non-IID Data and Time-varying Communication Links
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HyperPrism, a non-linear distributed mirror-descent aggregation framework that averages models via a weighted power mean with per-layer degrees chosen by hypernetworks, targeting non-IID data and time-varying links.
Problem
Distributed/decentralized machine learning suffers inter-model divergence from non-IID local data and time-varying communication links, which linear averaging aggregation handles poorly.
Approach
Perform aggregation in the mirror-descent dual space using Kolmogorov means (a weighted power mean with mapping w->w^p), with the power degree p chosen adaptively per layer and per round by a per-device hypernetwork, generalizing linear aggregation methods as special cases (p=1).
Key results
- Under Dirichlet=0.1, density=0.5, 50 devices, HyperPrism reaches 90.61% max accuracy on LR+MNIST in 6 rounds versus 11 for ADOM, with the paper reporting convergence-speed gains up to 98.63%.
- On CIFAR-10 CNN with Dirichlet=0.1, HyperPrism achieves 20.45% max accuracy versus 11.46% for ADOM and 20.16% for Mudag, while ADOM fails to converge (inf rounds).
- Scalability: with 100 devices on LR+MNIST, HyperPrism reaches 90.75% in 3 rounds while ADOM needs 42 rounds.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use HyperPrism instead of ADOM | Logistic Regression · MNIST, Dirichlet=0.1 | Max Acc: 90.58 -> 90.61 | Helped |
| use HyperPrism instead of ADOM | Logistic Regression · MNIST, Dirichlet=0.1 | Conv. Rds: 11 -> 6 | Helped |
| use HyperPrism instead of ADOM | Logistic Regression · MNIST, Dirichlet=1 | Max Acc: 90.88 -> 90.30 | Hurt |
| use HyperPrism instead of ADOM | Logistic Regression · MNIST, Dirichlet=1 | Conv. Rds: 10 -> 7 | Helped |
| use HyperPrism instead of ADOM | Logistic Regression · MNIST, Dirichlet=10 | Max Acc: 90.58 -> 90.60 | Helped |
| use HyperPrism instead of ADOM | Logistic Regression · MNIST, Dirichlet=10 | Conv. Rds: 23 -> 8 | Helped |
| use HyperPrism instead of SwarmSGD | CNN · CIFAR-10, Dirichlet=10 | Max Acc: 72.22 -> 75.74 | Helped |
| use HyperPrism instead of SwarmSGD | CNN · CIFAR-10, Dirichlet=10 | Conv. Rds: 14 -> 13 | Helped |
| use HyperPrism instead of Mudag | CNN · CIFAR-10, Dirichlet=0.1 | Max Acc: 20.16 -> 20.45 | Helped |
| use HyperPrism instead of Mudag | CNN · CIFAR-10, Dirichlet=0.1 | Conv. Rds: 83 -> 86 | Hurt |
| remove hypernetwork (ablation) | HyperPrism · LR + MNIST | Total (s): 41.72 -> 30.38(↓) | Helped |
| remove hypernetwork (ablation) | HyperPrism · CIFAR-10 | Total (s): 2074.79 -> 1974.76(↓) | Helped |
Surprising findings
- ADOM fails to converge within the round budget on CIFAR-10 CNN in most settings, and barely converges at 100 devices.
Limitations
- Added computational overhead from hypernetworks, with per-iteration time higher than baselines.
- Experiments limited to small models (logistic regression and a CNN) on MNIST and CIFAR-10.
- Accuracy fluctuates notably with the choice of p, so selecting p matters.
Experimental setup
Datasets: MNIST CIFAR-10
Benchmarks: MNIST CIFAR-10
Metrics: top-1 accuracy max accuracy convergence rounds wall-clock time
Baselines: DPSGD SwarmSGD Mudag ADOM
Builds on: distributed mirror descent Kolmogorov means HyperNetworks pFedLA DPSGD
Closest papers by meaning
- ATA: Adaptive Task Allocation for Efficient Resource Management in Distributed Machine Learning (ICML 2025 · similarity 0.86)
- Fault Tolerant ML: Efficient Meta-Aggregation and Synchronous Training (ICML 2024 · similarity 0.86)
- AdaGK-SGD: Adaptive Global Knowledge Guided Distributed Stochastic Gradient Descent (AAAI 2025 · similarity 0.86)
- TurboSVM-FL: Boosting Federated Learning through SVM Aggregation for Lazy Clients (AAAI 2024 · similarity 0.85)
- Robust Distributed Gradient Aggregation Using Projections onto Gradient Manifolds (AAAI 2024 · similarity 0.85)
- SparsyFed: Sparse Adaptive Federated Learning (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.