Research map / Architectures and efficiency / Spiking networks
NDOT: Neuronal Dynamics-based Online Training for Spiking Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes NDOT, a forward-in-time online training method for SNNs that decomposes the full gradient into spatial gradients and neuronal-dynamics-based temporal gradients, achieving constant training memory across time-steps.
Problem
Training deep spiking neural networks with surrogate gradients and BPTT requires storing intermediate states over all time-steps, causing memory growth linear in time-steps and precluding online, forward-in-time learning.
Approach
Uses a continuous temporal dependency e[t] = u(t+1)/u(t) derived from LIF neuronal dynamics to track temporal gradients recursively in the forward pass, combined with per-time-step spatial gradients and an instantaneous CE+MSE loss, so no backpropagation through time is needed; FTRL is used as intuition for the implicit use of history.
Key results
- NDOT-O reaches 94.89% on CIFAR-10 with VGG-11 (WS) at 6 time-steps, outperforming OTTT (93.73%) at the same time-steps.
- On CIFAR-100 at 6 time-steps, NDOT reaches 76.61% versus OTTT's 71.11%, a gain above 5 points.
- On CIFAR10-DVS at 10 time-steps, NDOT-O reaches 77.50% versus OTTT's 77.10%; GPU training memory stays constant across time-steps.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use NDOT instead of OTTT | VGG-11 (WS) · CIFAR-10, 6 time-steps | accuracy (%): 93.73 -> 94.89 | Helped |
| Use NDOT instead of OTTT | VGG-11 (WS) · CIFAR-100, 6 time-steps | accuracy (%): 71.11 -> 76.61 | Helped |
| Use NDOT instead of OTTT | VGG-11 (WS) · DVS-CIFAR10, 10 time-steps | accuracy (%): 77.10 -> 77.50 | Helped |
| Add the FTRL loss to OTTT | VGG-11 (WS) · CIFAR-100, 6 time-steps | accuracy (%): 71.11 -> 75.89 | Helped |
| Set α to 0.1 in the CE/MSE loss combination (ablation) | NDOTO · DVS-CIFAR10, T=6 | accuracy (%): 74.2 -> 75.6 | Helped |
| Set α to 1.0 in the CE/MSE loss combination (ablation) | NDOTO · DVS-CIFAR10, T=6 | accuracy (%): 75.6 -> 71.0 | Hurt |
Surprising findings
- NDOT with T=1 on CIFAR-10 (94.28%) already exceeds LTL with T=16 (93.20%).
Limitations
- Uses weight standardization instead of batch normalization, since BN across time is incompatible with online gradients.
- Relies on an approximate numerical-stability strategy when membrane potential minus threshold is near zero (Appendix A.2).
- Evaluated only on CIFAR-10, CIFAR-100 and CIFAR10-DVS with a VGG-11 architecture.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 CIFAR10-DVS
Benchmarks: CIFAR-10 CIFAR-100 CIFAR10-DVS
Metrics: top-1 accuracy GPU training memory
Baselines: OTTT FTRL-OTTT SLTT TET TEBN tdBN LTL QCFS SlipReLu PLIF
Builds on: OTTT BPTT FTRL TET SLTT
Closest papers by meaning
- Online Training Through Time for Spiking Neural Networks (NeurIPS 2022 · similarity 0.93)
- Temporal Flexibility in Spiking Neural Networks: Towards Generalization Across Time Steps and Deployment Friendliness (ICLR 2025 · similarity 0.90)
- Sign Gradient Descent-based Neuronal Dynamics: ANN-to-SNN Conversion Beyond ReLU Network (ICML 2024 · similarity 0.90)
- Advancing Training Efficiency of Deep Spiking Neural Networks through Rate-based Backpropagation (NeurIPS 2024 · similarity 0.90)
- Enhanced Self-Distillation Framework for Efficient Spiking Neural Network Training (NeurIPS 2025 · similarity 0.89)
- Adaptive Surrogate Gradients for Sequential Reinforcement Learning in Spiking Neural Networks (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.