Research map / Learning theory and algorithms / Prediction and decision losses
DeltaGrad: Rapid retraining of machine learning models
Yinjun Wu, Edgar Dobriban, Susan Davidson
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DeltaGrad instead of retraining from scratch for batch addition/deletion | Regularized logistic regression and two-layer neural network · MNIST, Covertype, HIGGS, RCV1 | accuracy: BaseL and DeltaGrad accuracies are equal or differ only slightly; confidence intervals overlap | No clear effect |
| Use DeltaGrad instead of retraining from scratch for batch addition/deletion | Regularized logistic regression · MNIST | speedup: up to 2.6x | Improved |
| Use DeltaGrad instead of retraining from scratch for batch addition/deletion | Regularized logistic regression · Covertype | speedup: 2x | Improved |
| Use DeltaGrad instead of retraining from scratch for batch addition/deletion | Regularized logistic regression · HIGGS | speedup: 1.6x | Improved |
| Use DeltaGrad instead of retraining from scratch for batch addition/deletion | Regularized logistic regression · RCV1 | speedup: 6.5x | Improved |
| Use DeltaGrad instead of retraining from scratch for online updates | Logistic regression · MNIST, Covertype, HIGGS, RCV1 | accuracy: BaseL and DeltaGrad prediction accuracies are identical in Table 2 | No clear effect |
| Use DeltaGrad instead of retraining from scratch for online updates | Logistic regression · MNIST | speedup: 2.5x | Improved |
| Use DeltaGrad instead of retraining from scratch for online updates | Logistic regression · Covertype | speedup: 2x | Improved |
| Use DeltaGrad instead of retraining from scratch for online updates | Logistic regression · HIGGS | speedup: 1.8x | Improved |
| Use DeltaGrad instead of retraining from scratch for online updates | Logistic regression · RCV1 | speedup: 6.5x | Improved |
| Use the non-strongly-convex DeltaGrad variant instead of retraining from scratch | Two-layer neural network · MNIST | speedup: about 1.4x | Improved |
Later papers that compared with this method
1 later accepted paper reports DeltaGrad as a baseline in their results tables; 1 of them report a better number on most of the results they share with it.
- Prompt Certified Machine Unlearning with Randomized Gradient Smoothing and Quantization (NeurIPS 2022 · better on 4 of 4 shared results)
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- When to retrain a machine learning model (ICML 2025 · similarity 0.85)
- LoTUS: Large-Scale Machine Unlearning with a Taste of Uncertainty (CVPR 2025 · similarity 0.84)
- Learning to Rewind via Iterative Prediction of Past Weights for Practical Unlearning (AAAI 2025 · similarity 0.84)
- Reproduce, Replicate, Reevaluate. The Long but Safe Way to Extend Machine Learning Methods (AAAI 2024 · similarity 0.83)
- Retraining with Predicted Hard Labels Provably Increases Model Accuracy (ICML 2025 · similarity 0.83)
- Self-Boost via Optimal Retraining: An Analysis via Approximate Message Passing (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
