Research map / Adversarial robustness and security / Attacks on text and graphs

SMI: Statistical Membership Inference for Reliable Unlearned Model Auditing

NeurIPS 2026 · Adversarial robustness and security › Attacks on text and graphs · approach: Membership inference and inversion · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SMI, a training-free unlearning auditor that estimates the non-member mixture proportion rho in the unlearned feature distribution (via covariance matching or MMD), with bootstrap reference ranges, instead of relying on shadow-model-based membership inference.

Problem

Auditing whether machine unlearning truly forgets data: MIA-based auditing treats samples classified as non-members as forgotten, which can yield illusory forgetting when attacks fail, and shadow-model MIAs are computationally expensive.

Approach

Reformulates unlearning auditing as estimating the mixture proportion rho such that the unlearned feature distribution is approximately (1-rho) member + rho non-member, using low-order moment matching (SMI) or kernel mean embedding / MMD (SMI-M), with bootstrap ranges and no shadow models or classifiers.

Key results

• On Resnet50-Cifar100-10%, SMI-M estimates rho* = 93.95% versus 74.44% for RULI and 87.38% for LiRA (best baseline).
• SMI (covariance-based) reaches 90.89% on Resnet50-Cinic10-10% versus 90.47% for LiRA and 89.88% for RULI.
• SMI runtime is 930.71 ms with 200 bootstraps, compared with 5672.83 ms for RULI and 8 shadow models plus 2.46e6 ms per shadow model for training.

Surprising findings

• Under adversarially forged unlearning the estimated rho* can greatly exceed 1 (reported >>10), so the mixture assumption can fail.
• Wasserstein-based variant (SMI-W) performed worse than the simple moment-matching SMI, per the authors' discussion.

Limitations

• Requires roughly 20 audit samples per class; error grows with smaller audit sets.
• Mixture assumption F_u ≈ (1-rho)F_m + rho F_n can break under adversarial unlearning.
• Evaluated only on image classification with retraining as target and three approximate methods; theoretical bounds are loosely connected to the auditing claim.

Experimental setup

Datasets: CIFAR-10 CIFAR-100 CINIC-10

Benchmarks: unlearning audit benchmark released with IAM (CIFAR-10, CIFAR-100, CINIC-10)

Metrics: estimated forgetting ratio rho* FNR FPR runtime bootstrap reference range

Baselines: RULI IAM RMIA EMIA LiRA Unleak Random

Builds on: LiRA RMIA IAM RULI PAC-Bayes

Closest papers by meaning

• The BatchNorm Illusion: Diagnosing Normalization Artifacts in Machine Unlearning Evaluation (NeurIPS 2026 · similarity 0.86)
• Adversarial Machine Unlearning (ICLR 2025 · similarity 0.86)
• When to Forget? Complexity Trade-offs in Machine Unlearning (ICML 2025 · similarity 0.86)
• A Reliable Cryptographic Framework for Empirical Machine Unlearning Evaluation (NeurIPS 2025 · similarity 0.85)
• Verification of Machine Unlearning is Fragile (ICML 2024 · similarity 0.85)
• Causal Evaluation of Membership Inference Attacks (NeurIPS 2026 · similarity 0.85)