Research map / Federated learning and privacy / Privacy attacks and secure learning
NoT: Federated Unlearning via Weight Negation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes NoT, a federated unlearning method that negates (multiplies by -1) the first layer's weights of the converged global model and then fine-tunes on retained data, with no extra storage or access to forget data.
Problem
Federated unlearning must remove a client's data contributions from a trained FL model without retraining from scratch, without auxiliary storage, and without access to the data to be forgotten.
Approach
Perturb the converged global model by negating weights of selected layers (mainly the first layer) to break inter-layer co-adaptation and raise loss, then fine-tune on retained data; a theoretical framework argues this perturbation is strong (C1) and layer-wise optimality preserving (C2) so recovery is fast.
Key results
- On ViT Caltech-101 client-wise unlearning, NoT reports an average gap of 0.79 versus Retrain's reference, with communication cost 0.79 (x10^12 bytes) and computation 6.31e20 FLOPs, the lowest among compared methods except FCU's gap of 1.61 (which is higher).
- On CIFAR-10 CNN client-wise unlearning, NoT achieves average gap 0.29 versus 0.81 to 4.09 for baselines (FT 1.01, FedEraser 1.58, FCU 0.81), with computation 3.06e16 FLOPs versus Retrain's 5.81e16.
- In centralized unlearning on CIFAR-10 ResNet-18, NoT reaches average gap 0.34 with 2.19e15 FLOPs, close to the best-gap method l1-sparse (0.19) which needs more than 30% more computation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use weight negation followed by fine-tuning instead of the strongest baseline | NoT · CIFAR-10, CNN, client-wise IID | Avg. Gap: 0.81 -> 0.29 | Helped |
| Use weight negation followed by fine-tuning instead of the strongest baseline | NoT · CIFAR-100, CNN, client-wise IID | Avg. Gap | Helped |
| Use weight negation followed by fine-tuning instead of the strongest baseline | NoT · CIFAR-10, ResNet-18, client-wise IID | Avg. Gap: 0.84 -> 1.78 | Hurt |
| Use weight negation followed by fine-tuning instead of the strongest baseline | NoT · CIFAR-100, ResNet-18, client-wise IID | Avg. Gap: 2.51 -> 4.73 | Hurt |
| Use weight negation followed by fine-tuning instead of the strongest baseline | NoT · Caltech-101, ViT-B/16, client-wise IID | Avg. Gap: 1.60 -> 0.79 | Helped |
| Use weight negation followed by fine-tuning rather than retraining | NoT · Caltech-101, ViT-B/16, client-wise IID | Communication cost: 1.76e12 -> 8.08e11 | Helped |
| Negate the convolution projection layer (ablation) | NoT on ViT-B/16 · Caltech-101 | Avg. Gap | Helped |
| Negate the positional embedding layer (ablation) | NoT on ViT-B/16 · Caltech-101 | Avg. Gap | Mixed |
| Negate the first self-attention head's query weights (ablation) | NoT on ViT-B/16 · Caltech-101 | Avg. Gap | Helped |
| Negate the second self-attention head's query weights (ablation) | NoT on ViT-B/16 · Caltech-101 | Avg. Gap | Hurt |
| Replace weight negation with random initialization (ablation) | NoT on the ViT convolution projection layer · Caltech-101 | Avg. Gap | Hurt |
| Replace weight negation with added white noise (ablation) | NoT on the ViT convolution projection layer · Caltech-101 | Avg. Gap | Hurt |
Surprising findings
- Weight negation of the first layer outperforms random re-initialization and adding noise, which perform similarly to plain fine-tuning.
- FedEraser and FUKD perform worse than retraining in the client-wise setting because they must refer back to the first checkpoint, making full retraining more efficient.
Limitations
- Requires access to retain data for fine-tuning recovery, and some useful knowledge is lost during negation.
- Evaluated only on small vision classification datasets (CIFAR-10/100, Caltech-101), not on larger datasets such as ImageNet.
- Theory for ReLU-like activations is not fully covered; approximate layer-wise optimality is conjectured.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 Caltech-101
Benchmarks: CIFAR-10 CIFAR-100 Caltech-101
Metrics: retain accuracy forget accuracy test accuracy MIA communication cost computation cost average gap
Baselines: Retrain FT FedEraser FUKD PGD MoDE FCU RandL GA BadT l1-sparse SSD
Builds on: FedAvg SSD SalUn l1-sparse unlearning FedEraser
Closest papers by meaning
- Federated Unlearning with Gradient Descent and Conflict Mitigation (AAAI 2025 · similarity 0.93)
- Unlearning through Knowledge Overwriting: Reversible Federated Unlearning via Selective Sparse Adapter (CVPR 2025 · similarity 0.89)
- Federated Learning with Extremely Noisy Clients via Negative Distillation (AAAI 2024 · similarity 0.88)
- FedWon: Triumphing Multi-domain Federated Learning Without Normalization (ICLR 2024 · similarity 0.87)
- Ferrari: Federated Feature Unlearning via Optimizing Feature Sensitivity (NeurIPS 2024 · similarity 0.87)
- FedLMT: Tackling System Heterogeneity of Federated Learning via Low-Rank Model Training with Theoretical Guarantees (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.