Research map / Federated learning and privacy / Differential privacy
DMM: Distributed Matrix Mechanism for Differentially-Private Federated Learning Based on Constant-Overhead Linear Secret Resharing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes the Distributed Matrix Mechanism (DMM), which uses a constant-overhead packed-secret-sharing resharing protocol (LRP) to apply correlated matrix-mechanism noise across committees in distributed (local) DP federated learning, and evaluates it on SO-NWP and FEMNIST against DDG and central BandMF.
Problem
Federated learning under distributed (local) differential privacy has lagged behind central DP because the noise cannot be correlated across rounds when participating clients change each iteration, so the matrix mechanism cannot be applied.
Approach
A linear packed secret resharing protocol (LRP) that moves secret-shared gradients and noise from one committee to the next with O(1) per-secret communication instead of n^2, letting clients form linear combinations of shares across iterations so that the server only sees matrix-mechanism outputs with correlated noise.
Key results
- On Stack Overflow next-word prediction, DMM improves accuracy over the distributed DDG baseline by about 5-6 percentage points and approaches central DP methods.
- On FEMNIST, DMM outperforms DDG by roughly 4 percentage points (abstract claims up to 14% at given privacy levels).
- With the Honaker factorization and committee size 64, per-iteration LRP client computation is 94.2 ms and communication 5.73 MB on FEMNIST, versus naive resharing at 2.61 GB.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use correlated noise across iterations instead of fresh noise each iteration | DMM versus DDG · Stack Overflow Next Word Prediction | test accuracy: 5-6 percentage points | Helped |
| use correlated noise across iterations instead of fresh noise each iteration | DMM versus DDG · Federated EMNIST | test accuracy: ≈ 4 percentage points | Helped |
| use DMM instead of prior local or distributed DP mechanisms | Federated learning · standard federated learning tasks | accuracy: up to 14% | Helped |
| use DMM instead of central DP BandMF or Honaker online mechanisms | Stack Overflow Next Word Prediction · Stack Overflow Next Word Prediction | test accuracy | Mixed |
| use Honaker online factorization instead of optimal factorization (ablation) | DMM · Stack Overflow Next Word Prediction and Federated EMNIST | test accuracy | Hurt |
| use Honaker online factorization instead of optimal factorization (ablation) | DMM · Federated EMNIST | communication per client per iteration: 828 MB -> 5.73 MB | Helped |
| use Honaker online factorization instead of optimal factorization (ablation) | DMM · Federated EMNIST | computation per client per iteration: 3.34 s -> 94.2 ms | Helped |
| replace naive secret resharing with LRP | DMM · Federated EMNIST | communication per client per iteration: 379 GB -> 5.73 MB | Helped |
| replace naive secret resharing with LRP | DMM · Stack Overflow Next Word Prediction | communication per client per iteration: 2.13 TB -> 25.1 MB | Helped |
| use DMM with Honaker online factorization instead of Secure Aggregation | DMM versus DDG · Federated EMNIST | communication per client per iteration: 5.73 MB vs 4.07 MB | Hurt |
Surprising findings
- The optimal matrix factorization adds substantial computation and communication (3.34 s, 828 MB on FEMNIST) for only a small accuracy gain over the Honaker online factorization.
- Skellam noise performed far worse empirically than the discrete Gaussian in preliminary experiments.
Limitations
- Security is only passive/information-theoretic for the main protocol; malicious security is left to future work and the protocol assumes t_c + t_d < (1/2 - mu)n.
- Only computational DP is obtained because secure aggregation is used, and the experiments do not verify the central-DP-level utility claim.
- Optimal matrix factorization incurs very high client cost and communication in the worst case.
Experimental setup
Datasets: Stack Overflow Next Word Prediction Federated EMNIST
Benchmarks: Stack Overflow Next Word Prediction Federated EMNIST
Metrics: test accuracy client computation time client communication cost privacy epsilon
Baselines: DDG (distributed discrete Gaussian) BandMF Honaker online matrix mechanism (central) Secure Aggregation (Flamingo) naive secret resharing
Builds on: matrix mechanism distributed discrete Gaussian mechanism Honaker online factorization Flamingo secure aggregation packed secret sharing
Closest papers by meaning
- Mitigating the Privacy–Utility Trade-off in Decentralized Federated Learning via f-Differential Privacy (NeurIPS 2025 · similarity 0.89)
- Privacy-Preserving Federated Convex Optimization: Balancing Partial-Participation and Efficiency via Noise Cancellation (ICML 2025 · similarity 0.87)
- Private and Federated Stochastic Convex Optimization: Efficient Strategies for Centralized Systems (ICML 2024 · similarity 0.87)
- Clients Collaborate: Flexible Differentially Private Federated Learning with Guaranteed Improvement of Utility-Privacy Trade-off (ICML 2025 · similarity 0.87)
- A New Federated Learning Framework Against Gradient Inversion Attacks (AAAI 2025 · similarity 0.87)
- Towards Trustworthy Federated Learning with Untrusted Participants (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.