Research map / Continual learning / Online and replay-based
Hebbian Learning based Orthogonal Projection for Continual Learning of Spiking Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HLOP, a continual learning method for spiking neural networks that uses lateral skew-symmetric connections trained with Hebbian/anti-Hebbian rules to extract principal subspaces of activity traces and project weight updates orthogonally to old-task subspaces, achieving near-zero forgetting.
Problem
Catastrophic forgetting in spiking neural networks during continual learning; existing orthogonal projection methods need SVD/matrix inversion on small batches and lack neuronal implementations.
Approach
Recurrent lateral connections with skew-symmetric weights, updated by Hebbian and anti-Hebbian learning on streaming data, extract the principal subspace of presynaptic activity traces; the lateral circuit then projects traces onto the orthogonal complement so weight updates do not interfere with old tasks. The approach is agnostic to the error propagation / SNN training method.
Key results
- On PMNIST (domain-incremental, online), HLOP reaches 95.15% ACC with -1.30% BWT versus 70.61% / -28.88% for the baseline with DSR.
- On 5-Datasets, HLOP achieves 88.65% ACC and -3.71% BWT, versus 79.70% for GPM and 79.66% for memory replay.
- On 10-split CIFAR-100, HLOP achieves 78.58% ACC with -0.26% BWT, outperforming GPM (77.48%) and EWC (73.75%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply HLOP to DSR | DSR SNN · PMNIST | ACC (%): 70.61 -> 95.15 | Helped |
| Apply HLOP to DSR | DSR SNN · PMNIST | BWT (%): $-28.88 -> $-1.30 | Helped |
| Apply HLOP to BPTT with SG | BPTT with SG SNN · PMNIST | ACC (%): 71.17 -> 95.08 | Helped |
| Apply HLOP to BPTT with SG | BPTT with SG SNN · PMNIST | BWT (%): $-28.23 -> $-1.54 | Helped |
| Apply HLOP to OTTT | OTTT SNN · PMNIST | ACC (%): 74.30 -> 94.76 | Helped |
| Apply HLOP to OTTT | OTTT SNN · PMNIST | BWT (%): $-24.86 -> $-1.56 | Helped |
| Apply HLOP instead of GPM | Continual SNN · 10-split CIFAR-100 | ACC (%): 77.48 -> 78.58 | Helped |
| Apply HLOP instead of GPM | Continual SNN · miniImageNet | BWT (%): $-$2.57 -> $-$0.48 | Helped |
| Apply HLOP instead of GPM | Continual SNN · 5-Datasets | ACC (%): 79.70 -> 88.65 | Helped |
| Apply HLOP instead of GPM | Continual SNN · 5-Datasets | BWT (%): $-$15.52 -> $-$3.71 | Helped |
| Use spike trains rather than weighted firing rate for Hebbian learning (ablation) | HLOP + DSR · PMNIST | ACC (%): 95.15 -> 95.23 | No clear effect |
| Use spike trains rather than weighted firing rate for Hebbian learning (ablation) | HLOP + DSR · 10-split CIFAR-100 | ACC (%): 78.58 -> 78.42 | No clear effect |
Surprising findings
- Applying HLOP to spike-based training with lateral circuits achieves similar results with discrete lateral spiking neurons of about 40 time steps on PMNIST.
- Hebbian learning over weighted firing rate versus full spike trains gives nearly identical results for DSR.
Limitations
- Lateral circuits use linear neurons in the main method; spiking lateral neurons require quantization approximation or ternary spiking that is not directly supported by common neuron models.
- Focuses on task- and domain-incremental settings; class-incremental learning would need replay combined with HLOP, which is not evaluated here.
Experimental setup
Datasets: PMNIST CIFAR-100 CIFAR-10 MNIST SVHN FashionMNIST notMNIST miniImageNet
Benchmarks: PMNIST 10-split CIFAR-100 5-Datasets 20-split miniImageNet
Metrics: average accuracy (ACC) backward transfer (BWT)
Baselines: Baseline (direct sequential training) Memory replay (MR) EWC HAT GPM
Builds on: GPM DSR OTTT Oja's rule feedback alignment
Closest papers by meaning
- Rethinking Hebbian Principle: Low-Dimensional Structural Projection for Unsupervised Learning (NeurIPS 2025 · similarity 0.90)
- Spike-timing-dependent Hebbian learning as noisy gradient descent (NeurIPS 2025 · similarity 0.85)
- Learning Successor Features with Distributed Hebbian Temporal Memory (ICLR 2025 · similarity 0.85)
- Two Tales of Single-Phase Contrastive Hebbian Learning (ICML 2024 · similarity 0.85)
- What Do Hebbian Learners Learn? Reduction Axioms for Iterated Hebbian Learning (AAAI 2024 · similarity 0.85)
- SPICED: A Synaptic Homeostasis-Inspired Framework for Unsupervised Continual EEG Decoding (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.