Research map / Architectures and efficiency / RNNs and normalization
Can Biologically Plausible Temporal Credit Assignment Rules Match BPTT for Neural Similarity? E-prop as an Example
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that e-prop, a truncation-based bio-plausible learning rule, reaches neural-similarity (Procrustes distance) to primate recordings comparable to BPTT at matched task accuracy, and that initialization gain can matter more than the learning rule.
Problem
Whether biologically plausible temporal credit assignment rules, besides matching task performance, produce representations as aligned with neural recordings as BPTT-trained RNNs.
Approach
Compares e-prop (and ModProp, node perturbation, evolution strategies) against BPTT on neural similarity using Procrustes distance at matched normalized accuracy on Mante 2013 and Sussillo 2015 primate datasets, and adds a 1D toy analysis showing e-prop convergence depends on initialization.
Key results
- At normalized accuracy ~0.8 on Sussillo 2015, BPTT and e-prop Procrustes distances are 0.398 and 0.407 (gain=1.0), with no significant difference (p=0.173).
- Initial weight gain changes Procrustes distance more than the learning rule: BPTT distance ranges 0.398 to 0.461 across gains 0 to 1.0.
- For Sussillo 2015 and Mante 2013, trained BPTT and e-prop models move closer to the data-data noise floor than untrained models, while Hatsopoulos 2007 BPTT matches the noise floor (p=0.311).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use e-prop instead of BPTT at matched accuracy | RNN · Sussillo 2015 | Procrustes distance: 0.406~$\pm$~0.012 -> 0.411~$\pm$~0.012 | No clear effect |
| Use e-prop instead of BPTT at matched accuracy | RNN · Mante 2013 | Procrustes distance: 0.721~$\pm$~0.008 -> 0.727~$\pm$~0.010 | No clear effect |
| Train the RNN rather than leave it untrained | BPTT RNN · Hatsopoulos 2007 | Procrustes distance: 0.538~$\pm$~0.041 -> 0.395~$\pm$~0.020 | Helped |
| Train the RNN rather than leave it untrained | BPTT RNN · Sussillo 2015 | Procrustes distance: 0.498~$\pm$~0.015 -> 0.406~$\pm$~0.012 | Helped |
| Train the RNN rather than leave it untrained | BPTT RNN · Mante 2013 | Procrustes distance: 0.800~$\pm$~0.008 -> 0.721~$\pm$~0.008 | Helped |
| Use e-prop instead of BPTT at matched accuracy | RNN · Sussillo 2015 | CCA: 0.272~$\pm$~0.009 -> 0.273~$\pm$~0.009 | No clear effect |
| Use e-prop instead of BPTT at matched accuracy | RNN · Sussillo 2015 | CKA: 0.152~$\pm$~0.015 -> 0.154~$\pm$~0.006 | No clear effect |
| Increase initial weight gain from 0.0 to 1.0 (ablation) | BPTT RNN · Sussillo 2015 | Procrustes distance: 0.461~$\pm$~0.006 -> 0.398~$\pm$~0.007 | Helped |
| Increase initial weight gain from 0.0 to 1.0 (ablation) | e-prop RNN · Sussillo 2015 | Procrustes distance: 0.461~$\pm$~0.007 -> 0.407~$\pm$~0.008 | Helped |
| Increase initial weight gain from 1.0 to 1.5 (ablation) | BPTT RNN · Sussillo 2015 | Procrustes distance: 0.398~$\pm$~0.007 -> 0.428~$\pm$~0.010 | Hurt |
| Increase initial weight gain from 1.0 to 1.5 (ablation) | e-prop RNN · Sussillo 2015 | Procrustes distance: 0.407~$\pm$~0.008 -> 0.432~$\pm$~0.008 | Hurt |
| Lower the BPTT learning rate from 1e-3 to 1e-4 (ablation) | BPTT RNN · Sussillo 2015 | distance to e-prop | Helped |
Surprising findings
- Initial weight gain affected neural distances more than the choice between BPTT and e-prop.
- E-prop typically requires more training iterations than BPTT to reach the same accuracy.
Limitations
- Only e-prop and ModProp among bio-plausible rules, and only two primate datasets with a fixed architecture, initialization and task.
- The pipeline cannot identify the brain's actual learning rule due to identifiability issues.
- Sussillo 2015 dataset cannot be redistributed; the comparison is limited to a small number of seeds and uncorrected t-tests.
Experimental setup
Datasets: Mante 2013 Sussillo 2015 Hatsopoulos 2007
Benchmarks: Mante 2013 Sussillo 2015 Hatsopoulos 2007
Metrics: Procrustes distance CCA CKA DSA normalized accuracy
Baselines: BPTT truncated BPTT node perturbation evolution strategies ModProp
Builds on: e-prop RFLO ModProp SnAp-n FORCE learning
Closest papers by meaning
- Beyond accuracy: generalization properties of bio-plausible temporal credit assignment rules (NeurIPS 2022 · similarity 0.89)
- Distinguishing Learning Rules with Brain Machine Interfaces (NeurIPS 2022 · similarity 0.88)
- Can the Brain Do Backpropagation? --- Exact Implementation of Backpropagation in Predictive Coding Networks (NeurIPS 2020 · similarity 0.86)
- Backpropagation at the Infinitesimal Inference Limit of Energy-Based Models: Unifying Predictive Coding, Equilibrium Propagation, and Contrastive Hebbian Learning (ICLR 2023 · similarity 0.86)
- Learning better with Dale’s Law: A Spectral Perspective (NeurIPS 2023 · similarity 0.86)
- Neural Representational Consistency Emerges from Probabilistic Neural-Behavioral Representation Alignment (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.