Research map / Federated learning and privacy / Privacy attacks and secure learning
SPEAR: Exact Gradient Inversion of Batches in Federated Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SPEAR, a sampling-based gradient inversion attack that exactly reconstructs batches (b>1) of inputs to the first ReLU fully connected layer in the honest-but-curious federated learning setting, using low-rank gradient structure and ReLU-induced sparsity.
Problem
Gradient inversion in federated learning: existing attacks recover inputs exactly only for batch size 1 and only approximately for larger batches in the honest-but-curious setting.
Approach
Exploits the explicit low-rank structure of weight gradients of a ReLU fully connected layer plus ReLU-induced sparsity of activation gradients; samples row-submatrices of a rank decomposition to obtain candidate directions via kernels, filters them by sparsity, then greedily selects directions maximizing a sparsity matching score, yielding exact input recovery.
Key results
- SPEAR recovers almost all batches exactly at b=20 across MNIST, CIFAR-10, TinyImageNet and ImageNet.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SPEAR instead of prior approximate attacks | ImageNet gradient inversion, Sigmoid network · ImageNet | PSNR: 38 -> 124 | Helped |
| Use SPEAR instead of prior attacks | ImageNet gradient inversion, ReLU network · ImageNet | PSNR: 19.6 -> 124 | Helped |
| Increase batch size from b=20 to b=24 | SPEAR on the narrow network (m=200) · Image reconstruction | accuracy: 99\% -> 63\% | Hurt |
| Increase network width | SPEAR · Image reconstruction | number of required iterations | Helped |
| Attack a later layer rather than the first layer | SPEAR on TinyImageNet batches, layer l=5 · TinyImageNet | runtime: 70 minutes/batch | Hurt |
| Train the network for 5000 gradient steps before inversion | SPEAR on layer l=5 · TinyImageNet | accuracy: >95\% | Helped |
| Add the second stage of the greedy reconstruction algorithm (ablation) | SPEAR · Image reconstruction across batch sizes | accuracy | Helped |
| Combine SPEAR with an optimization-based approximate attack to guide sampling | SPEAR · TinyImageNet, batches up to 100 | reconstruction quality | Helped |
| Use SPEAR-reconstructed features instead of CPA feature reconstructions | VGG16 feature inversion · ImageNet, b=16 | feature cosine similarity | Helped |
| Add DPSGD noise | SPEAR on TinyImageNet, b=20 · TinyImageNet | PSNR: >28 | Hurt |
| Vary the DP-SGD clipping constant C (ablation) | SPEAR · TinyImageNet, b=20 | reconstruction quality | No clear effect |
Surprising findings
- CI-Net's PSNR drops below 16 on ReLU networks compared with 38 on Sigmoid networks.
- Attacking layer 5 takes about 70 minutes per batch at initialization but under 1 minute after about 5000 training steps.
Limitations
- Exponential sampling cost in batch size, impractical beyond b≈25.
- Restricted to inputs of ReLU fully connected layers.
- Accuracy drops on narrower networks due to violated independence assumptions.
Experimental setup
Datasets: MNIST CIFAR-10 TinyImageNet ImageNet ADULT
Benchmarks: MNIST CIFAR-10 TinyImageNet ImageNet
Metrics: PSNR LPIPS accuracy
Baselines: Breaching
Builds on: sparsely-used dictionary learning
Closest papers by meaning
- DAGER: Exact Gradient Inversion for Large Language Models (NeurIPS 2024 · similarity 0.89)
- GRAIN: Exact Graph Reconstruction from Gradients (ICLR 2025 · similarity 0.88)
- Foreseeing Reconstruction Quality of Gradient Inversion: An Optimization Perspective (AAAI 2024 · similarity 0.87)
- Achieving Lossless Gradient Sparsification via Mapping to Alternative Space in Federated Learning (ICML 2024 · similarity 0.87)
- High-Fidelity Gradient Inversion in Distributed Learning (AAAI 2024 · similarity 0.86)
- Recovering Labels from Local Updates in Federated Learning (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.