Research map / Federated learning and privacy / Privacy attacks and secure learning

A Certified Unlearning Approach without Access to Source Data

ICML 2025 · Federated learning and privacy › Privacy attacks and secure learning · approach: Encryption-based private learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a certified unlearning method that removes forget data using only a surrogate dataset, calibrating Gaussian noise to an estimated statistical (KL-based) distance between surrogate and source distributions.

Problem

Certified machine unlearning when the original training (retain) data is unavailable and only a surrogate dataset with a different distribution can be used.

Approach

Estimate the source Hessian with the surrogate Hessian in a single-step Newton unlearning update, and scale Gaussian noise by a bound that depends on the total variation (approximated via KL divergence using energy-based SGLD samples and Donsker-Varadhan estimation) between surrogate and source distributions.

Key results

• On CIFAR-10 (xi=100), Unlearn (-) achieves 77.3% forget accuracy vs 77.8% for Retrain, with MIA 48.83% and RT 32 versus Retrain's 7 (concentration 100 column).
• On the mixed-linear CIFAR-10 class-0 unlearning setting, Unlearn (-) reaches 5.1% forget accuracy vs 0% for Retrain, with retain accuracy 93.5% vs 92.7%.
• Experiments show the surrogate-based method achieves utility comparable to methods with full source access across synthetic, real-world, and MNIST-USPS settings.

Measured effects reported in the paper

Surprising findings

• The paper notes that certified guarantees are only approximate in practice, since the statistical distance is estimated via KL divergence heuristics and hyperparameters (alpha, L, beta, gamma) are tuned per setting.

Limitations

• Theoretical guarantees assume exact statistical distance, which is only approximated in practice via KL divergence heuristics, giving potentially weaker guarantees.
• Convexity and smoothness assumptions do not hold for general neural networks; experiments rely on linear and mixed-linear models for the theory-aligned setting.
• Hyperparameters alpha, L, beta, gamma are tuned per setting, which may lead to approximate certification.

Experimental setup

Datasets: CIFAR-10 StanfordDogs Caltech256 MNIST USPS synthetic Gaussian

Benchmarks: CIFAR-10 StanfordDogs Caltech256 MNIST-USPS synthetic Gaussian

Metrics: train/test/retain/forget accuracy membership inference attack accuracy relearn time forget score

Baselines: Retrain Unlearn (+)

Builds on: certified unlearning Newton update (Sekhari et al.) Gaussian mechanism (differential privacy) Donsker-Varadhan representation implicit energy-based models (JEM) mixed-linear networks

Closest papers by meaning

• Certified Unlearning for Neural Networks (ICML 2025 · similarity 0.91)
• The Utility and Complexity of In- and Out-of-Distribution Machine Unlearning (ICLR 2025 · similarity 0.91)
• Towards Source-Free Machine Unlearning (CVPR 2025 · similarity 0.91)
• Unlearned but Not Forgotten: Data Extraction after Exact Unlearning in LLM (NeurIPS 2025 · similarity 0.90)
• Leveraging Per-Instance Privacy for Machine Unlearning (ICML 2025 · similarity 0.90)
• Efficient Source-free Unlearning via Energy-Guided Data Synthesis and Discrimination-Aware Multitask Optimization (ICML 2025 · similarity 0.89)