Research map / Optimization / Optimizers
Proximal Backpropagation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace explicit parameter updates with proximal updates | ProxProp versus BackProp on CIFAR-10 · CIFAR-10 test set | accuracy: ≈ 54 % -> ≈ 56 % | Helped |
| Use ProxProp update directions within Adam instead of BackProp directions | Convolutional network trained on CIFAR-10 · CIFAR-10 validation set | accuracy: 71.7% -> 72.9% | Helped |
| Use ProxProp update directions within Adam instead of BackProp directions | Convolutional network trained on CIFAR-10 · CIFAR-10 test set after 50 epochs | accuracy: 69.6% -> 70.7% | Helped |
| Use ProxProp update directions within Adam instead of BackProp directions | Convolutional network trained on CIFAR-10 · CIFAR-10 full batch training | training loss | Helped |
| Use proximal parameter updates rather than explicit BackProp updates | Fully connected network trained on CIFAR-10 · CIFAR-10 full batch training across step sizes | stability | Helped |
| Increase the number of conjugate-gradient iterations for the proximal update (ablation) | ProxProp on CIFAR-10 · CIFAR-10 full batch training | training loss | Helped |
| Use approximate proximal updates with three CG iterations instead of BackProp | Fully connected network trained on CIFAR-10 · CIFAR-10 full batch, 50 epochs, τ=0.05 | training loss: 0.091 -> 1.334 | Hurt |
| Use approximate proximal updates with five CG iterations instead of BackProp | Fully connected network trained on CIFAR-10 · CIFAR-10 full batch, 50 epochs, τ=0.05 | training loss: 0.091 -> 1.049 | Hurt |
| Use three or five CG iterations for proximal updates rather than explicit BackProp updates | Fully connected network trained on CIFAR-10 · CIFAR-10 full batch, 50 epochs, τ=1 | training stability: BackProp: --; ProxProp (cg3): $2\cdot 10^{-3}$; ProxProp (cg5): $3\cdot 10^{-4}$ | Helped |
Closest papers by meaning
- Beyond Backprop: Online Alternating Minimization with Auxiliary Variables (ICML 2019 · similarity 0.90)
- Understanding and Improving Optimization in Predictive Coding Networks (AAAI 2024 · similarity 0.86)
- LPGD: A General Framework for Backpropagation through Embedded Optimization Layers (ICML 2024 · similarity 0.86)
- Accelerated training through iterative gradient propagation along the residual path (ICLR 2025 · similarity 0.86)
- What's in a Prior? Learned Proximal Networks for Inverse Problems (ICLR 2024 · similarity 0.85)
- Reducing Fine-Tuning Memory Overhead by Approximate and Memory-Sharing Backpropagation (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.