Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning

Towards Source-Free Machine Unlearning

CVPR 2025 · Label-efficient and robust learning › Zero-/few-shot and meta-learning · approach: Zero-shot robustness and transfer · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a source-free machine unlearning method that estimates the retain-data Hessian from only the forget data and trained model, enabling zero-shot unlearning for linear and mixed-linear classifiers with bounded-error guarantees.

Problem

Machine unlearning when the original training (remaining) data is unavailable, so the retain Hessian needed for Newton-style unlearning cannot be computed directly.

Approach

Approximates the retain Hessian by solving a semidefinite program that matches second-order loss differences at random perturbed points, substituting forget-set loss differences for retain-set ones under a bounded-error assumption, and proves a Frobenius-norm error bound and a gradient-norm bound on the unlearned model.

Key results

• On CIFAR-10 with a linear classifier and 10% forget data, Unlearned(-) reaches 70% test / 71% remaining / 68% forget vs. 72/74/72 for Retrained, outperforming NegGrad, Random Labels, JiT, and Adversarial baselines (around 51-52% test).
• With 5% forget data on CIFAR-10, the performance gap to retraining is 0%, while at 20% on mixed-linear CIFAR-100 the gap reaches up to 6.6% on remaining data.
• Increasing the number of perturbations (250 to 1000) and the L2 regularization lambda (0 to 0.001) progressively closes the gap to the retrained model.

Measured effects reported in the paper

Limitations

• Theory applies to linear classifiers with convex losses; deep networks are handled only via linearization (mixed linear).
• Performance gap to retraining widens as forget-set size grows (e.g., 14% at 10% in the first CIFAR-10 table row set).
• Storing and inverting the full Hessian is computationally impractical for large models.

Experimental setup

Datasets: CIFAR-10 CIFAR-100 Stanford Dogs Caltech-256

Benchmarks: CIFAR-10 CIFAR-100 Stanford Dogs Caltech-256

Metrics: test accuracy remaining accuracy forget accuracy membership inference attack score

Baselines: Retrained Unlearned(+) NegGrad Random Labels JiT Adversarial

Builds on: Guo et al. certified data removal (Newton update) Mixed Linear Unlearning zero-shot machine unlearning Just-in-Time unlearning

Closest papers by meaning

• The Utility and Complexity of In- and Out-of-Distribution Machine Unlearning (ICLR 2025 · similarity 0.92)
• Hessian-Free Online Certified Unlearning (ICLR 2025 · similarity 0.91)
• Efficient Source-free Unlearning via Energy-Guided Data Synthesis and Discrimination-Aware Multitask Optimization (ICML 2025 · similarity 0.91)
• Certified Unlearning for Neural Networks (ICML 2025 · similarity 0.91)
• A Certified Unlearning Approach without Access to Source Data (ICML 2025 · similarity 0.91)
• Rewind-to-Delete: Certified Machine Unlearning for Nonconvex Functions (NeurIPS 2025 · similarity 0.91)