Research map / Optimization / Optimizers
Feedback Gradient Descent: Efficient and Stable Optimization with Orthogonality for DNNs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FGD instead of SGD with momentum | WideResNet 28-10 · CIFAR-10 | accuracy: 96.27 ± 0.054 -> 96.51 ± 0.062; 0.24% | Helped |
| Use FGD instead of SGD with momentum | WideResNet 28-10 · CIFAR-100 | accuracy: 81.47 ± 0.282 -> 82.25 ± 0.220; 0.78% | Helped |
| Use FGD instead of Cayley SGD | WideResNet 28-10 · CIFAR-100 | accuracy: 81.90 -> 82.25 ± 0.220 | Helped |
| Use FGD instead of SGD with momentum | ResNet18 · CIFAR-100 | accuracy: 77.85 -> 78.87; more than 1% | Helped |
| Use FGD instead of SGD with momentum | VGG13 · CIFAR-100 | accuracy: 74.27 -> 75.24; 0.97% | Helped |
| Use FGD instead of SGD with momentum | ResNet34 · ImageNet ILSVRC-2012 | top-1 accuracy: 0.47% | Helped |
| Use FGD instead of SGD with momentum | ResNet50 · ImageNet ILSVRC-2012 | top-1 accuracy: 76.13 -> 76.80; 0.67% | Helped |
| Apply orthogonality only to convolutional layers in the last two residual modules (ablation) | FGD and OCNN on ResNet · CIFAR-10/100 | accuracy | Helped |
Closest papers by meaning
- Classification at the Edge of Stability: Unifying Self-Stabilization and Convergence Rates (NeurIPS 2026 · similarity 0.86)
- Orthogonal Weight Normalization: Solution to Optimization Over Multiple Dependent Stiefel Manifolds in Deep Neural Networks (AAAI 2018 · similarity 0.86)
- Information Geometry of Orthogonal Initializations and Training (ICLR 2020 · similarity 0.86)
- Convergence Rates for Gradient Descent on the Edge of Stability for Overparametrised Least Squares (NeurIPS 2025 · similarity 0.85)
- No Wrong Turns: The Simple Geometry Of Neural Networks Optimization Paths (ICML 2024 · similarity 0.85)
- Error Feedback under $(L_0,L_1)$-Smoothness: Normalization and Momentum (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.