Research map / Architectures and efficiency / Pruning and efficient tuning
SEMU: Singular Value Decomposition for Efficient Machine Unlearning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SEMU, an SVD-based machine unlearning method that projects gradients of the forget set onto low-rank subspaces and trains only a small r×r matrix per layer, altering under 1-2% of weights and optionally needing no remaining data.
Problem
Machine unlearning for classifiers and diffusion generators typically alters large portions of the model and depends on the remaining dataset for stability, which is costly and often unavailable due to privacy constraints.
Approach
Compute the forget-set gradient per layer, remove its component along the existing weights, take a truncated SVD (rank chosen by explained-variance threshold gamma), and add a trainable r×r update restricted to that gradient subspace, so only a small number of parameters change and no remaining data is required.
Key results
- On CIFAR-100 ResNet-18 with 10% random forgetting, SEMU reaches TA 74.14 (Retrain 74.13) while training 1.18% of parameters.
- On CIFAR-10 ResNet-18 class-wise forgetting, SEMU uses 0.87% trainable parameters with UA 99.83 and TA 92.26 vs Retrain 92.47.
- On CIFAR-10 DDPM class-wise forgetting, SEMU (1.2% params) reaches UA 95.60 vs SalUn 99.20 at 50% params, with FID 16.93 vs SalUn 11.21.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use the full remaining dataset instead of no remaining data (ablation) | SEMU · CIFAR-10 airplane class unlearning, DDPM | UA: 95.60 -> 100.00 | Helped |
| use a small subset of remaining data instead of no remaining data (ablation) | SEMU · CIFAR-10 airplane class unlearning, DDPM | UA: 95.60 -> 99.40 | Helped |
| use a small subset of remaining data instead of no remaining data (ablation) | SEMU · CIFAR-10 airplane class unlearning, DDPM | FID: 16.93 -> 13.93 | Helped |
| use the full remaining dataset instead of no remaining data (ablation) | SEMU · CIFAR-10 airplane class unlearning, DDPM | FID: 16.93 -> 14.51 | Helped |
Surprising findings
- Access to the remaining dataset gives only slightly better results, suggesting it is not strictly necessary.
- SalUn degrades noticeably when the available forget-class data or saliency sparsity is reduced, while SEMU is reported as more robust.
Limitations
- Does not use remaining data to constrain changes in directions crucial to downstream tasks, which may hurt performance.
- Sample quality may degrade in some concept-unlearning cases when no remaining data is used.
- Evaluated only on vision classification and image diffusion; not yet on LLMs or VLMs.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 Imagenette I2P
Benchmarks: CIFAR-10 CIFAR-100 Imagenette I2P
Metrics: unlearning accuracy remaining accuracy testing accuracy membership inference attack FID trainable parameter percentage
Baselines: Retrain FT RL GA IU BE BS l1-sparse SalUn SalUn-soft ESD FMN
Builds on: SalUn random labeling unlearning Eckart-Young-Mirsky theorem ESD
Closest papers by meaning
- When to Forget? Complexity Trade-offs in Machine Unlearning (ICML 2025 · similarity 0.88)
- LoTUS: Large-Scale Machine Unlearning with a Taste of Uncertainty (CVPR 2025 · similarity 0.87)
- Efficient Utility-Preserving Machine Unlearning with Implicit Gradient Surgery (NeurIPS 2025 · similarity 0.87)
- DualOptim: Enhancing Efficacy and Stability in Machine Unlearning with Dual Optimizers (NeurIPS 2025 · similarity 0.87)
- Machine Unlearning under Overparameterization (NeurIPS 2025 · similarity 0.87)
- Towards Scalable Exact Machine Unlearning Using Parameter-Efficient Fine-Tuning (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.