Research map / Federated learning and privacy / Privacy attacks and secure learning
Unlearning through Knowledge Overwriting: Reversible Federated Unlearning via Selective Sparse Adapter
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FUSED, a reversible federated unlearning method that identifies knowledge-sensitive layers and trains independent sparse adapters on remaining data with the original model frozen, overwriting forgotten knowledge.
Problem
Federated unlearning must remove specific client, class, or sample knowledge without indiscriminately harming shared knowledge across clients, while remaining reversible and cheap compared with retraining.
Approach
Critical Layer Identification ranks layers by client-update divergence (Manhattan distance) and attaches sparse, randomly masked trainable adapters to the top-K layers; the original weights stay frozen and the adapters are trained only on retained data (knowledge overwriting), so removing adapters restores the original model.
Key results
- On CIFAR-10 ResNet18 client unlearning, FUSED achieves RA 0.67 and FA 0.05, close to Retraining (RA 0.71, FA 0.04).
- FUSED reduces communication to 0.98M per client versus 42.73M for Retraining on CIFAR-10 ResNet18 (per the table).
- In the knowledge interference test, FUSED's overlapping-knowledge accuracy (C-Acc) is 0.37, matching Retraining's 0.38 and well above Federaser's 0.14 and Exact-Fun's 0.07.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FUSED rather than retraining | LeNet · FashionMNIST client unlearning | Comp: 210.15 -> 158.96 | Helped |
| Use FUSED rather than retraining | ResNet18 · Cifar10 client unlearning | Comm: 42.73M -> 0.98M | Helped |
| Use FUSED rather than retraining | SimpleViT · Cifar10 client unlearning | Comp: 5388.83 -> 502.60 | Helped |
| Use FUSED rather than retraining | ResNet18 · Cifar100 client unlearning | Comm: 42.91M -> 0.98M | Helped |
| Use FUSED rather than retraining | ResNet18 · Cifar10 client unlearning | RA: 0.71 -> 0.67 | Hurt |
| Use FUSED rather than retraining | ResNet18 · Cifar10 client unlearning | FA: 0.04 -> 0.05 | Hurt |
| Use FUSED rather than retraining | SimpleViT · Cifar10 client unlearning | RA: 0.33 -> 0.41 | Helped |
| Use FUSED rather than retraining | FUSED · Cifar10 overlapping knowledge after client unlearning | C-Acc: 0.38 -> 0.37 | Hurt |
| Use CLI to select sensitive layers instead of selecting layers randomly (ablation) | FUSED · Cifar10 client unlearning | remaining-data accuracy | Helped |
| Use CLI to select sensitive layers instead of selecting layers randomly (ablation) | FUSED · Cifar10 sample unlearning | remaining-data accuracy | No clear effect |
Surprising findings
- On the Cifar10 Transformer client unlearning case, FUSED's RA (0.33) and FA (0.07) differ from Retraining's and the paper states it even surpasses Retraining there.
- Reducing the amount of retraining data by half leaves FUSED's RA/FA (0.65/0.03) largely unchanged.
Limitations
- Still requires a large amount of remaining data to train the adapters.
- Layer selection depends on a preset K and an early federated iteration; ablation gap is smaller for sample unlearning.
Experimental setup
Datasets: FashionMNIST CIFAR-10 CIFAR-100
Benchmarks: FashionMNIST CIFAR-10 CIFAR-100
Metrics: RA FA ReA MIA Comp Comm PS
Baselines: Retraining FedEraser Exact-Fun EraseClient
Builds on: FedAvg FedEraser Exact-Fun EraseClient
Closest papers by meaning
- NoT: Federated Unlearning via Weight Negation (CVPR 2025 · similarity 0.89)
- Federated Unlearning with Gradient Descent and Conflict Mitigation (AAAI 2025 · similarity 0.89)
- Efficient Verified Unlearning For Distillation (NeurIPS 2025 · similarity 0.88)
- Unlearning or Obfuscating? Jogging the Memory of Unlearned LLMs via Benign Relearning (ICLR 2025 · similarity 0.87)
- Efficient Source-free Unlearning via Energy-Guided Data Synthesis and Discrimination-Aware Multitask Optimization (ICML 2025 · similarity 0.87)
- Certified Unlearning for Neural Networks (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.