Research map / Architectures and efficiency / RNNs and normalization
To update or not to update? Neurons at equilibrium in deep models
Andrea Bragagnolo, Enzo Tartaglione, Marco Grangetto
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| set SGD momentum mu_opt to 0 instead of 0.9 (ablation) | NEq (neurons at equilibrium) on ResNet-32 · CIFAR-10 | accuracy: 92.96% -> 91.84% | Hurt |
| set SGD momentum mu_opt to 0 instead of 0.9 (ablation) | NEq on ResNet-32 · CIFAR-10 | back-propagation FLOPs | Improved |
| skip back-propagation for equilibrium neurons (naive implementation) | ResNet-18 · ImageNet-1K | back-propagation wall-clock time: -17.52% vs theoretical -23.08% | Improved |
| lower percentage of updated neurons u in a layer (ablation) | fully connected (512x1000) and convolutional (64x128x3x3) layers · layer back-prop timing, several batch sizes | back-propagation time | Improved |
| use a single validation image for the equilibrium computation (ablation) | NEq · classification | accuracy | No clear effect |
Closest papers by meaning
- Rotational Equilibrium: How Weight Decay Balances Learning Across Neural Networks (ICML 2024 · similarity 0.86)
- Improving Deep Learning Speed and Performance Through Synaptic Neural Balance (AAAI 2025 · similarity 0.85)
- Measure gradients, not activations! Enhancing neuronal activity in deep reinforcement learning (NeurIPS 2025 · similarity 0.84)
- Separation and Bias of Deep Equilibrium Models on Expressivity and Learning Dynamics (NeurIPS 2024 · similarity 0.84)
- Improving equilibrium propagation without weight symmetry through Jacobian homeostasis (ICLR 2024 · similarity 0.84)
- MAD-TD: Model-Augmented Data stabilizes High Update Ratio RL (ICLR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
