Research map / Federated learning and privacy / Differential privacy

Sketched Gaussian Mechanism for Private Federated Learning

NeurIPS 2025 · Federated learning and privacy › Differential privacy · approach: Differentially private federated learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces the Sketched Gaussian Mechanism (SGM), which jointly analyzes Gaussian sketching and Gaussian noise via Rényi DP to show sketching adds privacy, and applies it to federated learning (Fed-SGM) with convergence guarantees.

Problem

Private federated learning must jointly handle client-level differential privacy and per-round communication cost, but existing analyses treat sketching privacy and Gaussian-mechanism privacy in isolation.

Approach

Combine an isometric Gaussian sketch with the Gaussian mechanism and analyze the joint release with RDP; the sketch dimension b contributes privacy so epsilon scales as O(1/(sqrt(b) sigma_g^2)), with no upper limit on b, and Fed-SGM uses the sketch in FL with GD or AMSGrad at the server.

Key results

• Fed-SGM attains the same privacy guarantee with lower Gaussian noise than unsketched DP-FedAvg; e.g., noise scale 0.0883 vs 0.8 for epsilon about 2.75 on EMNIST (vision task)
• Fed-SGM with Adam outperforms unsketched baselines in the reported experiments, and matches or exceeds unsketched GD variant on both tasks
• Convergence bound scales logarithmically in ambient dimension d and linearly in absolute intrinsic Hessian dimension

Measured effects reported in the paper

Surprising findings

• Sketching itself provides privacy amplification, so the sketched mechanism needs strictly less Gaussian noise than the unsketched mechanism at the same epsilon
• Adam-based Fed-SGM with sketching outperforms the GD variant without sketching

Limitations

• Analysis is limited to isotropic Gaussian sketching matrices
• Experiments are limited to two tasks with small client counts and a single seed setup as described
• Convergence bounds rely on assumptions such as bounded gradients and sub-Gaussian noise and a Hessian intrinsic-dimension assumption

Experimental setup

Datasets: EMNIST ByClass SST-2 (GLUE)

Benchmarks: EMNIST ByClass SST-2

Metrics: test accuracy training accuracy privacy epsilon Gaussian noise scale

Baselines: DP-FedAvg DiffSketch Adam-based unsketched DP FL

Builds on: DP-FedAvg DP-SGD Renyi differential privacy FetchSGD DiffSketch

Closest papers by meaning

• The Gaussian Mixing Mechanism: Renyi Differential Privacy via Gaussian Sketches (NeurIPS 2025 · similarity 0.89)
• Noisy SIGNSGD Is More Differentially Private Than You (Might) Think (ICML 2025 · similarity 0.89)
• Privacy-Aware Compression for Federated Learning Through Numerical Mechanism Design (ICML 2023 · similarity 0.89)
• Sketching for First Order Method: Efficient Algorithm for Low-Bandwidth Channel and Vulnerability (ICML 2023 · similarity 0.89)
• Clients Collaborate: Flexible Differentially Private Federated Learning with Guaranteed Improvement of Utility-Privacy Trade-off (ICML 2025 · similarity 0.89)
• Privacy-Preserving Federated Convex Optimization: Balancing Partial-Participation and Efficiency via Noise Cancellation (ICML 2025 · similarity 0.88)