Research map / Label-efficient and robust learning / Recognition and long-tail
Towards Eliminating Hard Label Constraints in Gradient Inversion Attacks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes an analytical algorithm that recovers augmented (label-smoothed or mixup) soft labels and last-layer input features from single-input gradients by searching a scalar lambda that minimizes a variance loss, then uses the recovered features to reconstruct images from fully-connected networks.
Problem
Gradient inversion attacks have only been evaluated under hard one-hot label constraints; analytical label recovery fails for soft labels produced by label smoothing and mixup, and analytical image reconstruction from unbiased fully-connected networks is unsolved for multi-class tasks.
Approach
Rewrite the label entries as functions of the last-layer gradient and a scalar lambda_r (replacing the unknown input with lambda_r times the gradient), then choose lambda_r by minimizing the variance of all non-top label entries (top-one for label smoothing, top-two for mixup), with L-BFGS first and PSO as fallback. Also shows a necessary condition: a label-distribution feature is needed to pick t
Key results
- Label recovery accuracy above 95% for label smoothing and mixup on CIFAR-100, Flowers-17 and ImageNet with ResNet50 (e.g., 99.2% and 100.0% on CIFAR-100).
- On CIFAR-10 with LeNet, recovered labels reach 99-100% accuracy across untrained and trained networks; image reconstruction with recovered labels matches ground-truth-label performance.
- With last-layer feature recovery, FCN-4 image reconstruction reaches average PSNR of 51.30 (label smoothing) and 66.80 (mixup) on 100 random samples.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Recover augmented labels and last-layer features analytically | Proposed label recovery · CIFAR-100, Flowers-17, and ImageNet | label recovery accuracy: 99.2; 99.1; 100.0 | Helped |
| Use the proposed method instead of iDLG for one-hot label recovery | ResNet50 label recovery · CIFAR-100, Flowers-17, and ImageNet | accuracy: 100.0 vs 100.0; 95.6 vs 95.6; 99.9 vs 99.9 | No clear effect |
| Use recovered labels rather than ground-truth labels for image reconstruction | Image reconstruction on LeNet · CIFAR-10 validation, mixup and label smoothing | PSNR, SSIM, and LPIPS: Under all experimental settings, recovering images with recovered labels is almost identical to recovering with ground-t | No clear effect |
| Initialize DLG with recovered labels | DLG image reconstruction · CIFAR-10 validation, cosine similarity loss | image reconstruction quality: 20.42 | Helped |
| Add last-layer feature loss to gradient matching (ablation) | Gradient inversion image reconstruction · Mixup and label smoothing; L2 and cosine matching | PSNR: 29.04 -> 34.09; 25.71 -> 28.75; 20.49 -> 24.96; 23.02 -> 25.79 | Helped |
| Set feature-loss weight to 2 rather than 1 (ablation) | Gradient inversion image reconstruction · Mixup and label smoothing; L2 and cosine matching | PSNR: 34.09 -> 34.18; 28.75 -> 28.81; 24.96 -> 24.65; 25.79 -> 25.85 | Mixed |
| Add last-layer gradient loss alongside feature loss (ablation) | Gradient inversion image reconstruction · Mixup and label smoothing; L2 and cosine matching | PSNR: 34.09 -> 29.28; 28.75 -> 25.89; 24.96 -> 21.00; 25.79 -> 23.27 | Hurt |
| Replace variance loss with squared-label loss (ablation) | Label recovery · CIFAR-10 label smoothing, untrained and trained ResNet18 | accuracy: 100%; 97.78% | Mixed |
| Add Gaussian or Laplace noise with variance up to 10^-2 to last-layer gradients (ablation) | Proposed label recovery · CIFAR-10 validation, label smoothing, untrained ResNet18 | accuracy: 100; 100; 100 | No clear effect |
| Increase Gaussian or Laplace noise variance to 10^-1 (ablation) | Proposed label recovery · CIFAR-10 validation, label smoothing, untrained ResNet18 | accuracy: 45; 36 | Hurt |
Surprising findings
- iDLG cannot guarantee 100% accuracy in the one-hot setting; the failure occurs when the network predicts an input with extreme one-hot probability.
- Simply satisfying sum-to-one and non-negativity of the pseudo label is insufficient to identify the true label; a label distribution prior is necessary.
Limitations
- Label recovery relies on exact ground-truth gradients; accuracy drops sharply under Gaussian or Laplace noise with variance 0.1 (about 45% and 36% accuracy).
- Fails for extreme one-hot predictions and for out-of-bound lambda values, and requires known label-distribution features (variance loss) tailored to each augmentation type.
- Method applies to last-layer gradients of a fully-connected layer and the augmentations studied (label smoothing, mixup); other augmentations require a new loss.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 Flowers-17 ImageNet
Benchmarks: CIFAR-10 CIFAR-100 Flowers-17 ImageNet
Metrics: label recovery accuracy L1 label loss PSNR SSIM LPIPS
Baselines: iDLG DLG IG R-GAP
Builds on: iDLG DLG R-GAP Inverting gradients
Closest papers by meaning
- Recovering Labels from Local Updates in Federated Learning (ICML 2024 · similarity 0.90)
- High-Fidelity Gradient Inversion in Distributed Learning (AAAI 2024 · similarity 0.87)
- A Sample-Level Evaluation and Generative Framework for Model Inversion Attacks (AAAI 2025 · similarity 0.86)
- Be Careful What You Smooth For: Label Smoothing Can Be a Privacy Shield but Also a Catalyst for Model Inversion Attacks (ICLR 2024 · similarity 0.86)
- Soften to Defend: Towards Adversarial Robustness via Self-Guided Label Refinement (CVPR 2024 · similarity 0.85)
- SPEAR: Exact Gradient Inversion of Batches in Federated Learning (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.