Research map / Architectures and efficiency / RNNs and normalization
Toward Practical Equilibrium Propagation: Brain-inspired Recurrent Neural Network with Feedback Regulation and Residual Connections
Zhuo Liu, Tao Chen
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add cross-layer residual connections (ablation) | 10-hidden-layer symmetric EP RNN · MNIST | test accuracy (%): 92.49% -> 97.49% | Improved |
| add cross-layer residual connections (ablation) | 10-hidden-layer symmetric EP RNN · CIFAR-10 | test accuracy (%): 34.90% -> 44.46% | Improved |
| add random skip-layer connections (AGT) (ablation) | 10-hidden-layer asymmetric EP RNN · MNIST | test accuracy (%): 87.37% -> 96.87% | Improved |
| add random skip-layer connections (AGT) (ablation) | 10-hidden-layer asymmetric EP RNN · CIFAR-10 | test accuracy (%): 30.37% -> 30.94% | No clear effect |
| increase feedback scaling beta from 0.01 to 1 (ablation) | 3-hidden-layer symmetric EP RNN · MNIST | test accuracy (%): 97.64% -> 55.86% | Hurt |
| increase feedback scaling beta from 0.01 to 1 (ablation) | 10-hidden-layer symmetric EP RNN · MNIST | test accuracy (%): 89.99% -> 14.07% | Hurt |
| increase feedback scaling beta from 0.1 to 1 (ablation) | 10-hidden-layer residual symmetric EP RNN · MNIST | test accuracy (%): 97.46% -> 95.51% | Hurt |
| lower feedback scaling beta from 0.1 to 0.001 (ablation) | 5-hidden-layer symmetric EP RNN · MNIST | test accuracy (%): 97.60% -> 93.54% | Hurt |
| increase feedback scaling beta from 0.01 to 1 (ablation) | convolutional symmetric EP RNN · MNIST | test accuracy (%): 99.15% -> 11.35% | Hurt |
| use same learning rate for all layers instead of P-EP layer-specific rates (ablation) | 3-hidden-layer EP RNN (tanh, SGD) · MNIST | test accuracy (%): 97.65% -> 97.83% | Improved |
| use Adam optimizer (ablation) | 3-hidden-layer EP RNN (tanh) · MNIST | test accuracy (%): 97.83% -> 98.36% | Improved |
| use feedback-regulated EP RNN instead of P-EP | 2-hidden-layer network · MNIST | test accuracy (%): 98.05% -> 98.39% | Improved |
Closest papers by meaning
- Feedback control guides credit assignment in recurrent neural networks (NeurIPS 2024 · similarity 0.88)
- Forward Learning with Top-Down Feedback: Empirical and Analytical Characterization (ICLR 2024 · similarity 0.88)
- Improving equilibrium propagation without weight symmetry through Jacobian homeostasis (ICLR 2024 · similarity 0.87)
- Efficient Target Propagation by Deriving Analytical Solution (AAAI 2024 · similarity 0.87)
- Training a Predictive Coding Network on ImageNet using Equilibrium Propagation (NeurIPS 2026 · similarity 0.86)
- Unconditional stability of a recurrent neural circuit implementing divisive normalization (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
