Research map / Optimization / Optimizers
GGD: Grafting Gradient Descent
Yanjing Feng, Yongdao Zhou
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use grafting gradient instead of Adam's sampled gradient (GAdam) | Adam · MNIST | min squared L2 norm of full gradient: 5.1588 -> 5.2779 (x1e-3) | Hurt |
| use grafting gradient instead of mini-batch SGD gradient (GGD) | MBSGD · MNIST | min squared L2 norm of full gradient: 5.3185 -> 5.8986 (x1e-3) | Hurt |
| use grafting gradient instead of SVRG gradient (GSVRG) | SVRG · MNIST | min squared L2 norm of full gradient: 0.13407 -> 0.15203 (x1e-3) | Hurt |
| use grafting gradient instead of Adam's sampled gradient (GAdam) | Adam · CIFAR-10 | min squared L2 norm of full gradient: 1.6565 -> 1.4659 (x1e-4) | Improved |
| use grafting gradient instead of mini-batch SGD gradient (GGD) | MBSGD · CIFAR-10 | min squared L2 norm of full gradient: 634.79 -> 2681.0 (x1e-4) | Hurt |
| use grafting gradient instead of SVRG gradient (GSVRG) | SVRG · CIFAR-10 | min squared L2 norm of full gradient: 2.0849 -> 3.7647 (x1e-4) | Hurt |
| use GGD instead of vanilla SGD | vanilla SGD · ijcnn1, a9a, covtype, rcv1 (logistic regression) | train loss | Improved |
| use GGD with diminishing stepsize instead of SGD-IS or MBSGD | SGD-IS / MBSGD · ijcnn1, a9a, covtype, rcv1 (logistic regression) | train loss | Mixed |
| use GGD-WR-Adam instead of Adam | Adam · ijcnn1, a9a, covtype, rcv1 (logistic regression) | train loss | No clear effect |
| use GGD-WR-SVRG instead of SVRG | SVRG · covtype (logistic regression) | train loss | No clear effect |
| use GGD-WR-SVRG instead of SVRG | SVRG · ijcnn1 (logistic regression) | train loss | Improved |
| use GGD, GAdam, GSVRG instead of MBSGD, Adam, SVRG | MBSGD / Adam / SVRG · MNIST (LeNet-5) | train loss | Improved |
Closest papers by meaning
- Stochastic Reweighted Gradient Descent (ICML 2022 · similarity 0.88)
- Can Gaussian Sketching Converge Faster on a Preconditioned Landscape? (ICML 2024 · similarity 0.88)
- Stochastic Approximate Gradient Descent via the Langevin Algorithm (AAAI 2020 · similarity 0.88)
- New Insight into Hybrid Stochastic Gradient Descent: Beyond With-Replacement Sampling and Convexity (NeurIPS 2018 · similarity 0.87)
- Painless Stochastic Gradient: Interpolation, Line-Search, and Convergence Rates (NeurIPS 2019 · similarity 0.87)
- Incremental Gradient Descent with Small Epoch Counts is Surprisingly Slow on Ill-Conditioned Problems (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
