Research map / Architectures and efficiency / RNNs and normalization
Learning to Rewind via Iterative Prediction of Past Weights for Practical Unlearning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes InvWNN, a learned past-weight predictor that inverts Weight Nowcasting Networks and is applied iteratively after fine-tuning on the forget set to unlearn data using only the forget set.
Problem
Machine unlearning when the retained training data is unavailable or too costly to use, so unlearning must rely only on the forget set without degrading model performance or introducing incorrect knowledge.
Approach
Reverse the usual use of weight prediction (forecasting future weights) to predict less-trained past weights from a short forward fine-tuning trajectory on the forget set, using a pretrained DNN (InvWNN) that extends Task Vector's linear extrapolation to non-linear, and applies this iteratively to gradually erase forget-set knowledge.
Key results
- On CIFAR10 unlearning 50% of data, the proposed method reached RA 92.24% and TA 85.04% versus retrain at 91.45% TA, with UA 8.31% vs 8.11% for retrain.
- In the label-noise setting, the proposed method was the only one (with Task Vector) to increase test accuracy during unlearning, per Fig. 5 and Fig. 3(a).
- In the MNIST diffusion task, the proposed method reduced the ratio of generated '4' images while keeping KID low, and produced no '4'-like outputs while Task Vector produced many abnormal images.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use iterative InvWNN past-weight prediction instead of Task Vector rewinding | ResNet18 · CIFAR10, randomly selected 50% forget data | test accuracy (TA): 83.29±0.82 -> 85.04±1.10 | Helped |
| Use iterative InvWNN past-weight prediction instead of Task Vector rewinding | ResNet18 · CIFAR10, randomly selected 50% forget data | remaining accuracy (RA): 92.03±0.72 -> 92.24±0.71 | Helped |
| Apply Random Label, Boundary Unlearning, or SalUn to label-noisy forget data | ResNet18 · CIFAR10 label noise | test accuracy | Hurt |
| Use NegGrad or Task Vector rewinding on label-noisy forget data | ResNet18 · CIFAR10 label noise | test accuracy | Mixed |
| Use iterative InvWNN past-weight prediction on label-noisy forget data | ResNet18 · CIFAR10 label noise | test accuracy | Helped |
| Apply Random Label or SalUn to overfitted data | ResNet18 · CIFAR10, 1% training data | test loss | Helped |
| Apply Boundary Unlearning, NegGrad, or Task Vector to overfitted data | ResNet18 · CIFAR10, 1% training data | test loss | Hurt |
| Use iterative InvWNN past-weight prediction on overfitted data | ResNet18 · CIFAR10, 1% training data | test loss | Helped |
| Use NegGrad or Task Vector on backdoor data | ResNet18 · CIFAR10 BadNet backdoor | accuracy on forget data with correct labels | Helped |
| Use Random Label, Boundary Unlearning, or SalUn on backdoor data | ResNet18 · CIFAR10 BadNet backdoor | accuracy on forget data with correct labels | Hurt |
| Use iterative InvWNN past-weight prediction on backdoor data | ResNet18 · CIFAR10 BadNet backdoor | accuracy on forget data with correct labels | Helped |
| Use iterative InvWNN past-weight prediction on label-noisy data | PVTv2 · CIFAR100 label noise | test accuracy | Helped |
Surprising findings
- Random-label and SalUn-style methods that inject incorrect labels can further degrade performance and sometimes act as noise injection that partially mitigates overfitting.
- Task Vector and NegGrad generate abnormal images in diffusion unlearning, indicating high damage despite removing the target digit.
Limitations
- Termination condition requires matching retrained model's unlearning accuracy, which needs a retrained reference to set.
- Side-effect evaluation protocol is limited to CIFAR10/ResNet18 controlled settings plus a few additional tasks.
- InvWNN relies on a pretrained predictor trained on prior weight-history data from a specific prior work.
Experimental setup
Datasets: CIFAR10 CIFAR100 TinyImageNet MNIST weight histories from Jang et al. 2023
Benchmarks: CIFAR10 CIFAR100 TinyImageNet MNIST
Metrics: unlearning accuracy remaining accuracy test accuracy membership inference attack test loss backdoor attack success rate KID ratio of generated images classified as 4
Baselines: Random Label Random Label+SalUn Boundary Unlearning NegGrad Task Vector SalUn Retrain
Builds on: Weight Nowcasting Network Task Vector NegGrad SalUn
Closest papers by meaning
- Rewind-to-Delete: Certified Machine Unlearning for Nonconvex Functions (NeurIPS 2025 · similarity 0.89)
- Certified Unlearning for Neural Networks (ICML 2025 · similarity 0.89)
- When to Forget? Complexity Trade-offs in Machine Unlearning (ICML 2025 · similarity 0.89)
- Towards Source-Free Machine Unlearning (CVPR 2025 · similarity 0.89)
- The Utility and Complexity of In- and Out-of-Distribution Machine Unlearning (ICLR 2025 · similarity 0.88)
- LoTUS: Large-Scale Machine Unlearning with a Taste of Uncertainty (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.