Research map / Label-efficient and robust learning / OOD and anomaly detection
Factor Decorrelation Enhanced Data Removal from Deep Predictive Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DecoRemoval, combining random-Fourier-feature factor decorrelation with a linear loss perturbation and Newton-style certified removal, to keep accuracy under out-of-distribution data removal.
Problem
Data removal (machine unlearning) from deep predictive models often causes distribution shift and accuracy loss, especially out-of-distribution, while retraining is costly.
Approach
Applies random Fourier feature mapping with learned sample weights to decorrelate features, then performs certified-removal Newton updates on the last linear layer after adding a random linear term to the loss.
Key results
- On CGSS with 10000 removed samples, DecoRemoval reports F1 0.495 versus 0.477 for the best prior baseline cited in text (Zhang) and 0.471 for Foster.
- On ESS with 1000 removed samples, DecoRemoval reports 54.9% ACC versus 55.4% for Retrain, while Certified Removal gets 48.6%.
- Reported running time on MNIST with 1000 removed samples is 30.4 s for DecoRemoval versus 21.3 s for Certified Removal and 4643 s for Retrain.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DecoRemoval instead of CU | MLP data removal · MNIST OOD, 10000 removed samples | accuracy: 46.532 -> 48.340 | Helped |
| Use DecoRemoval instead of CU | MLP data removal · MNIST OOD, 10000 removed samples | F1: 0.450 -> 0.473 | Helped |
| Use DecoRemoval instead of CU | MLP data removal · CIFAR-10 OOD, 10000 removed samples | accuracy: 46.106 -> 47.832 | Helped |
| Use DecoRemoval instead of CU | MLP data removal · CIFAR-10 OOD, 10000 removed samples | F1: 0.444 -> 0.469 | Helped |
| Use DecoRemoval instead of DP-SGD | MLP data removal · SST-2 OOD, 10000 removed samples | accuracy: 90.101 -> 90.375 | Helped |
| Use DecoRemoval instead of DP-SGD | MLP data removal · SST-2 OOD, 10000 removed samples | F1: 0.816 -> 0.821 | Helped |
| Use DecoRemoval instead of CU | MLP data removal · ESS OOD, 10000 removed samples | accuracy: 50.871 -> 54.640 | Helped |
| Use DecoRemoval instead of SSD | MLP data removal · ESS OOD, 10000 removed samples | F1: 0.478 -> 0.510 | Helped |
| Use DecoRemoval instead of CU | MLP data removal · CGSS OOD, 10000 removed samples | accuracy: 47.851 -> 50.104 | Helped |
| Use DecoRemoval instead of CU | MLP data removal · CGSS OOD, 10000 removed samples | F1: 0.477 -> 0.495 | Helped |
| Vary hidden-layer size (ablation) | DecoRemoval · ESS, CGSS, SST-2, and MNIST OOD | accuracy and F1 | Mixed |
| Increase RFF dimension (ablation) | DecoRemoval · OOD datasets | accuracy and F1 | Helped |
Surprising findings
- Certified Removal is the fastest method in the efficiency table, faster than DecoRemoval on every dataset, contrary to the claim of superior efficiency.
Limitations
- Focuses only on feature-level decorrelation; integration with augmentation, adversarial training, or ensembles is unexplored.
- Certified removal may be suboptimal where high predictive accuracy is critical.
- Privacy guarantees are not formally certified for the full deep network; experiments focus on utility and efficiency.
Experimental setup
Datasets: MNIST CIFAR-10 SST-2 ESS CGSS
Benchmarks: MNIST CIFAR-10 SST-2 ESS CGSS
Metrics: ACC F1 running time membership inference attack success rate
Baselines: Retrain Certified Removal SISA DP-SGD SSD Certified Unlearning
Builds on: certified removal random Fourier features certified unlearning SSD
Closest papers by meaning
- Distribution-Level Feature Distancing for Machine Unlearning: Towards a Better Trade-off Between Model Utility and Forgetting (AAAI 2025 · similarity 0.85)
- Let Samples Speak: Mitigating Spurious Correlation by Exploiting the Clusterness of Samples (CVPR 2025 · similarity 0.83)
- Disentanglement Beyond Static vs. Dynamic: A Benchmark and Evaluation Framework for Multi-Factor Sequential Representations (NeurIPS 2025 · similarity 0.83)
- Machine Unlearning under Overparameterization (NeurIPS 2025 · similarity 0.83)
- Outlier Gradient Analysis: Efficiently Identifying Detrimental Training Samples for Deep Learning Models (ICML 2025 · similarity 0.83)
- Decomposing and Editing Predictions by Modeling Model Computation (ICML 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.