Research map / Federated learning and privacy / Privacy attacks and secure learning
Federated Unlearning with Gradient Descent and Conflict Mitigation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FedOSD, a federated unlearning method that replaces gradient ascent with a bounded unlearning cross-entropy loss, computes an orthogonal steepest descent update avoiding conflicts with retained clients, and uses gradient projection in post-training to prevent model reverting.
Problem
Federated unlearning of a target client via gradient ascent suffers gradient explosion, utility degradation from gradient conflicts with retained clients, and model reverting (recovering forgotten data) during post-training.
Approach
Use a bounded Unlearning Cross-Entropy loss minimized by gradient descent, pick the update direction closest to the inverse target gradient subject to being orthogonal to all retained clients' gradients (closed form via SVD of GG^T), and project retained gradients orthogonal to the direction back toward the original model during post-training.
Key results
- On FMNIST Pat-20 after post-training, FedOSD achieves ASR 0.023 with R-Acc 0.851 while FedRecovery, EWCSGA and FUPGA revert to ASR 0.685-0.960 (e.g., EWCSGA 0.935).
- On CIFAR-10 Pat-50 (m=10), FedOSD unlearns to ASR 0.000 and R-Acc 0.602 after unlearning, and 0.659 R-Acc with ASR 0.016 after post-training.
- Ablation: replacing the orthogonal direction with negative target gradient (M2) yields R-Acc 0.317 on FMNIST vs 0.806 for FedOSD after unlearning.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use FedOSD rather than FUPGA during unlearning | Federated client unlearning · FMNIST Pat-20 | R-Acc: .227(.254) -> .757(.187) | Helped |
| use FedOSD rather than FUPGA during unlearning | Federated client unlearning · CIFAR-10 Pat-20 | R-Acc: .202(.373) -> .549(.185) | Helped |
| add UCE instead of using Gradient Ascent on CE (ablation) | FedOSD · FMNIST after unlearning | R-Acc: .163 -> .806 | Helped |
| add UCE instead of using Gradient Ascent on CE (ablation) | FedOSD · CIFAR-10 after unlearning | R-Acc: .224 -> .602 | Helped |
| use the orthogonal steepest direction rather than -g_u (ablation) | FedOSD · FMNIST after unlearning | R-Acc: .317 -> .806 | Helped |
| use the orthogonal steepest direction rather than -g_u (ablation) | FedOSD · CIFAR-10 after unlearning | R-Acc: .391 -> .602 | Helped |
| use orthogonal steepest descent rather than a common descent direction (ablation) | FedOSD · FMNIST after unlearning | ASR: .835 -> .000 | Helped |
| use orthogonal steepest descent rather than a common descent direction (ablation) | FedOSD · CIFAR-10 after unlearning | ASR: .331 -> .000 | Helped |
| choose the closest feasible orthogonal direction instead of a random one (ablation) | FedOSD · FMNIST after unlearning | ASR: .046 -> .000 | Helped |
| choose the closest feasible orthogonal direction instead of a random one (ablation) | FedOSD · CIFAR-10 after unlearning | ASR: .009 -> .000 | Helped |
| add gradient projection during post-training (ablation) | FedOSD · FMNIST after post-training | ASR: .244$^r$ -> .021 | Helped |
| add gradient projection during post-training (ablation) | FedOSD · CIFAR-10 after post-training | ASR: .340$^r$ -> .024 | Helped |
Surprising findings
- Gradient-ascent-based baselines (EWCSGA, FUPGA) reduce FMNIST R-Acc to near random-model levels (about 0.1-0.25) during unlearning.
- Post-training with standard methods can increase ASR, meaning recovery of utility re-exposes the unlearned backdoor data.
Limitations
- Gradient projection in post-training relies on the distance to the original model omega^0 as a reference, limiting applicability when omega^0 is unavailable or drifts.
- SFU replication was highly time-consuming due to SVD of high-dimensional representation matrices.
- Evaluated only on small image classification benchmarks with simple CNN/MLP/ResNet models.
Experimental setup
Datasets: MNIST FMNIST CIFAR-10 CIFAR-100
Benchmarks: MNIST FMNIST CIFAR-10 CIFAR-100
Metrics: ASR R-Acc NC (conflicting clients count) worst/best R-Acc
Baselines: Retraining FedEraser FedRecovery MoDe EWCSGA FUPGA SFU
Builds on: Gradient Ascent unlearning FedEraser FedRecovery MoDe SFU
Closest papers by meaning
- NoT: Federated Unlearning via Weight Negation (CVPR 2025 · similarity 0.93)
- Unlearning through Knowledge Overwriting: Reversible Federated Unlearning via Selective Sparse Adapter (CVPR 2025 · similarity 0.89)
- Ferrari: Federated Feature Unlearning via Optimizing Feature Sensitivity (NeurIPS 2024 · similarity 0.87)
- Towards the Theory of Unsupervised Federated Learning: Non-asymptotic Analysis of Federated EM Algorithms (ICML 2024 · similarity 0.87)
- Certified Machine Unlearning via Noisy Stochastic Gradient Descent (NeurIPS 2024 · similarity 0.87)
- Achieving Dimension-Free Communication in Federated Learning via Zeroth-Order Optimization (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.