Research map / Architectures and efficiency / RNNs and normalization
Dendritic Localized Learning: Toward Biologically Plausible Algorithm
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Dendritic Localized Learning (DLL), a pyramidal-neuron-inspired algorithm with trainable backward weights and local soma-computed errors that satisfies three biological plausibility criteria while approaching backpropagation on MLPs, CNNs, and RNNs.
Problem
Backpropagation is biologically implausible due to weight symmetry, global error signals, and two-stage training; existing bio-plausible alternatives either violate these criteria or perform poorly.
Approach
Model a pyramidal neuron with basal (sensory input), apical (expected value / backpropagated activity) and soma (local error = x - u) compartments; replace transposed forward weights with a separately trainable backward weight Theta, and update forward and backward weights simultaneously so inference and learning are not two-stage.
Key results
- DLL-MLPs reach 97.57% on MNIST and DLL-CNNs reach 98.87%, versus 98.62% and 99.56% for backpropagation, averaged over 4 seeds.
- Among methods satisfying all three criteria, DLL achieves the highest average accuracy (71.89% MLP, 86.61% CNN), while Hebbian, R-STDP, Forward Forward and Equilibrium Propagation are lower or fail to converge on CNNs.
- On Harry Potter next-character prediction, DLL reaches 33.7% accuracy versus 51.9% for backpropagation; on Electricity DLL MSE is 0.172 vs 0.175 for backpropagation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Update Θ rather than keep the backward weights fixed and random (ablation) | DLL MLP · MNIST | accuracy: 97.37% -> 97.57% | Helped |
| Update Θ rather than keep the backward weights fixed and random (ablation) | DLL CNN · CIFAR-10 | accuracy: 69.85% -> 70.89% | Helped |
| Update Θ rather than keep the backward weights fixed and random (ablation) | DLL RNN · Harry Potter | accuracy: 0.71%* -> 33.70% | Helped |
| Update Θ rather than keep the backward weights fixed and random (ablation) | DLL RNN · Electricity | MSE: 0.193 -> 0.172 | Helped |
| Update Θ rather than keep the backward weights fixed and random (ablation) | DLL RNN · Electricity | MAE: 0.345 -> 0.321 | Helped |
| Update Θ rather than keep the backward weights fixed and random (ablation) | DLL RNN · Pems-bay | MSE: 0.198 -> 0.178 | Helped |
| Update Θ rather than keep the backward weights fixed and random (ablation) | DLL RNN · Pems-bay | MAE: 0.251 -> 0.224 | Helped |
| Use DLL instead of backpropagation | RNN · Harry Potter | prediction accuracy: 51.9% ± 1.0% -> 33.7% ± 0.6% | Hurt |
| Use DLL instead of backpropagation | RNN · Electricity | MSE: 0.175 ± 0.007 -> 0.172 ± 0.018 | Helped |
| Use DLL instead of backpropagation | RNN · Metr-la | MSE: 0.131 ± 0.004 -> 0.155 ± 0.005 | Hurt |
Surprising findings
- On the Electricity dataset, DLL-trained RNNs match or slightly beat backpropagation on some metrics.
- Forward Forward CNNs fail to converge (about 10% accuracy) on all benchmarks.
Limitations
- No formal convergence proof; convergence is supported empirically only.
- Performance still trails backpropagation on harder tasks such as CIFAR-10 and Harry Potter prediction.
- Experiments are limited to small-scale datasets and architectures.
Experimental setup
Datasets: MNIST FashionMNIST SVHN CIFAR-10 Harry Potter series Electricity Metr-la Pems-bay
Benchmarks: MNIST FashionMNIST SVHN CIFAR-10 Harry Potter next-character prediction Electricity Metr-la Pems-bay
Metrics: classification accuracy prediction accuracy MSE MAE
Baselines: Backpropagation Feedback Alignment Local Losses Predictive Coding Perturbation Learning Difference Target Propagation Hebbian Learning R-STDP Forward Forward Equilibrium Propagation
Builds on: feedback alignment difference target propagation predictive coding equilibrium propagation forward-forward algorithm
Closest papers by meaning
- Nonlinear dynamics of localization in neural receptive fields (NeurIPS 2024 · similarity 0.87)
- Complex priors and flexible inference in recurrent circuits with dendritic nonlinearities (ICLR 2024 · similarity 0.86)
- Dendritic Integration Inspired Artificial Neural Networks Capture Data Correlation (NeurIPS 2024 · similarity 0.85)
- Two Tales of Single-Phase Contrastive Hebbian Learning (ICML 2024 · similarity 0.85)
- Normative Networks for Source Separation via Local Plasticity and Dendritic Computation (NeurIPS 2026 · similarity 0.85)
- The Expressive Leaky Memory Neuron: an Efficient and Expressive Phenomenological Neuron Model Can Solve Long-Horizon Tasks. (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.