Research map / Federated learning and privacy / Privacy attacks and secure learning
Foreseeing Reconstruction Quality of Gradient Inversion: An Optimization Perspective
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LAVP, a loss-aware vulnerability proxy given by the max/min Hessian eigenvalue of the gradient matching loss at the ground truth, to predict gradient inversion reconstruction quality better than gradient norm for L2 and cosine losses.
Problem
Gradient inversion attacks in federated learning reconstruct private images from shared gradients, but the commonly used gradient norm vulnerability proxy is independent of the gradient matching loss (L2 or cosine) and so cannot explain loss-dependent differences in reconstruction quality.
Approach
Derives a loss-aware proxy from theory on gradient matching optimization near the ground truth: the maximum or minimum Hessian eigenvalue of the matching loss at x*, which equals J^T J for L2 and a projected variant for cosine; a loss-agnostic fusion takes the geometric mean of L2 and cosine LAVPs.
Key results
- On low-resolution datasets, LAVP for L2 (max eigenvalue) reaches |Spearman| around 0.5-0.66 with MSE versus about 0.35 for gradient norm on CIFAR-10 with L2.
- For cosine distance, the minimum-eigenvalue LAVP (ang_min) achieves Spearman up to about 0.75-0.8 with MSE on CIFAR-100, ImageNette and ImageWoof, while gradient norm stays near zero (e.g., -0.13 to 0.10).
- On ImageNet, LAVP for L2 (min eigenvalue) reaches 0.74 Spearman with MSE versus 0.66 for gradient norm on ResNet18/ShuffleNet.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace gradient norm with the L2 maximum Hessian eigenvalue proxy | Gradient inversion vulnerability ranking · CIFAR-10, L2 | Absolute Spearman correlation with MSE: 0.35 -> 0.51 | Helped |
| Replace gradient norm with the cosine minimum Hessian eigenvalue proxy | Gradient inversion vulnerability ranking · CIFAR-10, cosine | Absolute Spearman correlation with MSE: -0.28 -> -0.64 | Helped |
| Replace gradient norm with the L2 maximum Hessian eigenvalue proxy | Gradient inversion vulnerability ranking · CIFAR-100, L2 | Absolute Spearman correlation with MSE: 0.41 -> 0.46 | Helped |
| Replace gradient norm with the cosine minimum Hessian eigenvalue proxy | Gradient inversion vulnerability ranking · CIFAR-100, cosine | Absolute Spearman correlation with MSE: 0.10 -> -0.69 | Helped |
| Replace gradient norm with the L2 minimum Hessian eigenvalue proxy | Gradient inversion vulnerability ranking · ImageNette, L2 | Absolute Spearman correlation with MSE: 0.03 -> 0.34 | Helped |
| Replace gradient norm with the cosine minimum Hessian eigenvalue proxy | Gradient inversion vulnerability ranking · ImageNette, cosine | Absolute Spearman correlation with MSE: -0.13 -> -0.75 | Helped |
| Replace gradient norm with the L2 minimum Hessian eigenvalue proxy | Gradient inversion vulnerability ranking · ImageWoof, L2 | Absolute Spearman correlation with MSE: 0.35 -> 0.68 | Helped |
| Replace gradient norm with the cosine minimum Hessian eigenvalue proxy | Gradient inversion vulnerability ranking · ImageWoof, cosine | Absolute Spearman correlation with MSE: 0.00 -> -0.75 | Helped |
| Replace gradient norm with the L2 minimum Hessian eigenvalue proxy | Gradient inversion vulnerability ranking · ImageNet, L2 | Absolute Spearman correlation with MSE: 0.66 -> 0.74 | Helped |
| Replace gradient norm with the cosine maximum Hessian eigenvalue proxy | Gradient inversion vulnerability ranking · ImageNet, cosine | Absolute Spearman correlation with MSE: -0.06 -> 0.27 | Helped |
| Fuse L2 maximum and cosine minimum Hessian eigenvalue proxies using their geometric mean | Gradient inversion vulnerability ranking · CIFAR-10, L2 | Absolute Spearman correlation with MSE: 0.35 -> 0.48 | Helped |
| Fuse L2 maximum and cosine minimum Hessian eigenvalue proxies using their geometric mean | Gradient inversion vulnerability ranking · CIFAR-100, cosine | Absolute Spearman correlation with MSE: 0.10 -> 0.50 | Helped |
Surprising findings
- Gradient norm correlates with reconstruction MSE in the opposite direction for L2 and cosine losses (e.g., -0.28 vs 0.35 on CIFAR-10), showing it is not a consistent vulnerability proxy.
- Higher gradient-norm samples on ImageNet with L2 loss were not more vulnerable, which contradicts the prior intuition that larger gradient norm means greater vulnerability.
Limitations
- Analysis is limited to pure gradient matching loss without batch statistics matching, which state-of-the-art attacks use.
- Theorems assume L greater than a threshold epsilon, so small-Lipschitz outliers are not explained.
- Correlations are moderate (mostly |sigma| around 0.5 to 0.8) and vary across datasets and losses.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 ImageNette ImageWoof ImageNet
Benchmarks: CIFAR-10 CIFAR-100 ImageNette ImageWoof ImageNet
Metrics: Spearman correlation with MSE SSIM PSNR LPIPS
Baselines: gradient norm
Builds on: gradient inversion attack (Geiping et al.) Jacobian norm sensitivity proxy
Closest papers by meaning
- Concealing Sensitive Samples against Gradient Leakage in Federated Learning (AAAI 2024 · similarity 0.90)
- Gradient Inversion Attacks on Parameter-Efficient Fine-Tuning (CVPR 2025 · similarity 0.88)
- Leak and Learn: An Attacker's Cookbook to Train Using Leaked Data from Federated Learning (CVPR 2024 · similarity 0.87)
- High-Fidelity Gradient Inversion in Distributed Learning (AAAI 2024 · similarity 0.87)
- A New Federated Learning Framework Against Gradient Inversion Attacks (AAAI 2025 · similarity 0.87)
- SPEAR: Exact Gradient Inversion of Batches in Federated Learning (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.