Research map / Optimization / Nonconvex and smooth optimization
Improving Neural Ordinary Differential Equations with Nesterov's Accelerated Gradient Method
Nghia Nguyen, Tan Minh Nguyen, Võ Thục Khánh Huyền, Stanley Osher, Thieu Vo
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace NODE with GNesterovNODE | NODE · CIFAR10 | test accuracy: 0.5466 ± 0.0051 -> 0.6172 ± 0.0064 | Improved |
| replace NODE with GNesterovNODE | NODE · MNIST | test accuracy: 0.9531 ± 0.0042 -> 0.9807 ± 0.0013 | Improved |
| replace HBNODE with NesterovNODE | HBNODE · CIFAR10 | test accuracy: 0.5989 ± 0.0035 -> 0.5996 ± 0.0033 | No clear effect |
| replace GHBNODE with GNesterovNODE | GHBNODE · CIFAR10 | test accuracy: 0.6085 ± 0.0050 -> 0.6172 ± 0.0064 | Improved |
| replace GHBNODE with GNesterovNODE | GHBNODE · MNIST | test accuracy: 0.9817 ± 0.0005 -> 0.9807 ± 0.0013 | No clear effect |
| replace SONODE with NesterovNODE | SONODE · MNIST | test accuracy: 0.9824 ± 0.0013 -> 0.9824 ± 0.0015 | No clear effect |
| add bounded activation and skip connection (NesterovNODE to GNesterovNODE) (ablation) | NesterovNODE · CIFAR10 | test accuracy: 0.5996 ± 0.0033 -> 0.6172 ± 0.0064 | Improved |
| add bounded activation and skip connection (NesterovNODE to GNesterovNODE) (ablation) | NesterovNODE · MNIST | test accuracy: 0.9824 ± 0.0015 -> 0.9807 ± 0.0013 | No clear effect |
| increase Nesterov factor r from 3 to 5 (ablation) | GNesterovNODE · CIFAR10 | test accuracy: 0.6180 -> 0.6192 | Improved |
| increase Nesterov factor r from 5 to 10 (ablation) | GNesterovNODE · CIFAR10 | test accuracy: 0.6192 -> 0.6296 | Improved |
| increase Nesterov factor r (ablation) | GNesterovNODE · CIFAR10 | NFEs | Hurt |
| Euler solver with large step size 0.1/0.2/0.5: GNesterovNODE vs NODE (ablation) | NODE · CIFAR10 | training stability | Improved |
Closest papers by meaning
- A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights (JMLR 2016 · similarity 0.89)
- Global Convergence in Neural ODEs: Impact of Activation Functions (ICLR 2025 · similarity 0.86)
- How Deep Do We Need: Accelerating Training and Inference of Neural ODEs via Control Perspective (ICML 2024 · similarity 0.86)
- Provable Acceleration of Nesterov's Accelerated Gradient for Asymmetric Matrix Factorization and Linear Neural Networks (NeurIPS 2024 · similarity 0.86)
- From Fourier to Neural ODEs: Flow Matching for Modeling Complex Systems (ICML 2024 · similarity 0.86)
- Nyström-Accelerated Primal LS-SVMs: Breaking the $O(an^3)$ Complexity Bottleneck for Scalable ODEs Learning (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
