Research map / Architectures and efficiency / RNNs and normalization

The BatchNorm Illusion: Diagnosing Normalization Artifacts in Machine Unlearning Evaluation

NeurIPS 2026 · Architectures and efficiency › RNNs and normalization · approach: Batch normalization improvements · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that a weight-preserving BatchNorm recalibration pass over retain data can reverse apparent forgetting in approximate machine unlearning evaluations on CIFAR-style ResNets, and proposes it as a diagnostic control.

Problem

Approximate machine unlearning is evaluated with forget accuracy and linear probes, but on BatchNorm-based architectures the reported forgetting can be an artifact of running-statistics state rather than of the weights.

Approach

A single no-gradient forward pass over retain data that recomputes BN running statistics (weights unchanged) can shift forget accuracy by up to 78pp; the paper formalizes this as a weight-preserving fixed-point operator, attributes any pre/post gap to BN state, and decomposes linear-probe elevation into BN-measurement-bias and encoder-geometry terms.

Key results

• BN recalibration raises BadTeacher forget-class accuracy on CIFAR-10 from 19% to 97% (+78pp) with weights unchanged; six of nine methods show gaps of at least 18.9pp.
• With GroupNorm in place of BatchNorm, the forget-accuracy gap collapses to 0.00pp for all eight tested methods.
• Ten unlabeled images recover most masked forget accuracy, e.g., BadTeacher reaches 88.7% at B=10 in the adversarial recovery experiment.

Surprising findings

• On CIFAR-100, GA, SalUn and SSD show large negative BN-residuals: surface linear probe sits below retrain while the recalibrated probe exceeds it by 14-17pp.
• Membership inference AUC changes by at most 0.012 under recalibration despite large forget-accuracy changes.

Limitations

• Evaluation is limited to classification benchmarks with mostly class-level forget sets; no language models tested.
• Full ImageNet-1k evaluation was not feasible due to compute.
• The Pythagorean decomposition fails empirically, so the attribution is algebraic rather than quantitative in feature space.

Experimental setup

Datasets: CIFAR-10 CIFAR-100 Tiny-ImageNet ImageNet-100

Benchmarks: CIFAR-10 CIFAR-100 Tiny-ImageNet ImageNet-100

Metrics: forget accuracy retain accuracy linear probe accuracy nearest-class-mean accuracy relearning AUC membership inference AUC

Baselines: Retrain SCRUB GA GA+FT SalUn BadTeacher IncompTeacher NoiseInject SSD PGU

Builds on: SWA update_bn SCRUB SalUn BadTeacher

Closest papers by meaning

• LoTUS: Large-Scale Machine Unlearning with a Taste of Uncertainty (CVPR 2025 · similarity 0.91)
• When to Forget? Complexity Trade-offs in Machine Unlearning (ICML 2025 · similarity 0.90)
• Machine Unlearning under Overparameterization (NeurIPS 2025 · similarity 0.90)
• From Dormant to Deleted: Tamper-Resistant Unlearning Through Weight-Space Regularization (NeurIPS 2025 · similarity 0.89)
• DualOptim: Enhancing Efficacy and Stability in Machine Unlearning with Dual Optimizers (NeurIPS 2025 · similarity 0.89)
• Unified Gradient-Based Machine Unlearning with Remain Geometry Enhancement (NeurIPS 2024 · similarity 0.88)