Research map / Architectures and efficiency / Spiking networks
High-Performance Temporal Reversible Spiking Neural Networks with $\mathcal{O}(L)$ Training Memory and $\mathcal{O}(1)$ Inference Cost
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes T-RevSNN, a temporally reversible spiking architecture that keeps temporal dynamics only at stage-output spiking neurons, splits the encoded input across timesteps, and achieves O(L) training memory and O(1) inference cost.
Problem
Multi-timestep SNN training requires O(L×T) memory via BPTT, and multi-timestep inference costs O(T) energy; existing methods optimize only one of training or inference cost.
Approach
Turn off temporal dynamics in most spiking neurons (keeping them only at the last spiking layer of each stage), make the remaining temporal transfer reversible so membrane potentials need not be stored across timesteps, encode the image once and split features into T groups fed to T independent sub-networks that share parameters and exchange information only at key neurons, and use a ConvNeXt-styl
Key results
- On ImageNet-1K, T-RevSNN ResNet-18 (512) reaches 73.2% Top-1 with 29.8M params, 85.7 MB/img memory, 9.1 min/epoch, and 2.8 mJ energy, versus 73.4% for Spikformer-8-512 and 74.6% for Spike-driven Transformer.
- Compared with the spike-driven Transformer, the paper reports 8.6x memory efficiency, 2.0x training-time acceleration and 1.6x inference energy efficiency.
- On CIFAR10-DVS and DVS128 Gesture, T-RevSNN (16 timesteps) reaches 79.2% and 97.9% with 18.9 MB/img memory on Gesture.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use T-RevSNN ResNet-18 (512) instead of MS-ResNet-34 | ImageNet-1k · ImageNet-1k | Top-1 accuracy (%): 69.4 -> 73.2 | Helped |
| Use T-RevSNN ResNet-18 (512) instead of Spike-driven Transformer | ImageNet-1k · ImageNet-1k | Top-1 accuracy (%): 74.6 -> 73.2 | Hurt |
| Use T-RevSNN at 16 timesteps rather than Spikformer | CIFAR10-DVS · CIFAR10-DVS | Top-1 accuracy: 80.6 -> 79.2 | Hurt |
| Use T-RevSNN at 16 timesteps rather than Spike-Driven Transformer | DVS128 Gesture · DVS128 Gesture | Top-1 accuracy: 99.3 -> 97.9 | Hurt |
| Vary timestep from 2 to 16 (ablation) | T-RevSNN · ImageNet-1k | Accuracy: 68.1, 68.6, 69.8, 63.4 | Mixed |
| Vary timestep from 2 to 16 (ablation) | T-RevSNN · ImageNet-1k | Memory (MB/img): 75.6, 57.5, 49.8, 42.4 | Helped |
| Vary timestep from 2 to 16 (ablation) | T-RevSNN · ImageNet-1k | Training time (min/ep): 4.9, 6.1, 9.9, 16.0 | Hurt |
| Add multi-level temporal reversible feature fusion (ablation) | T-RevSNN · ImageNet-1k | Accuracy: 67.4 -> 68.6 | Helped |
| Add scaled residual connection (ablation) | T-RevSNN · ImageNet-1k | Accuracy: 68.3 -> 68.6 | Helped |
| Add scaled residual connection (ablation) | T-RevSNN · ImageNet-1k | Epochs to 60% accuracy: 32 -> 25 | Helped |
| Apply temporal reversibility to MS-ResNet-34 (ablation) | MS-ResNet-34 · ImageNet-1k | Accuracy: 68.3 -> 66.7 | Hurt |
| Apply temporal reversibility to MS-ResNet-34 (ablation) | MS-ResNet-34 · ImageNet-1k | Memory (MB/img): 267.1 -> 88.1 | Helped |
Surprising findings
- Applying temporal reversibility directly to MS-ResNet-34 drops accuracy from 68.3% to 66.7%, while memory falls from 267.1 to 88.1 MB/img.
- Replacing the spiking CNN block with spiking Transformer blocks in prior designs failed to converge, according to the authors.
Limitations
- Accuracy trails Spike-driven Transformer (73.2% vs 74.6%) and the Transformer-based variant was not realized.
- Evaluated only on ResNet-style models at moderate scale; larger models and long-sequence tasks are untested.
- Energy numbers are theoretical estimates rather than hardware measurements; some baseline energies are estimated from spiking ResNet.
Experimental setup
Datasets: ImageNet-1K CIFAR10-DVS DVS128 Gesture
Benchmarks: ImageNet-1K CIFAR10-DVS DVS128 Gesture
Metrics: Top-1 accuracy training memory (MB/img) training time (min/epoch) inference energy (mJ) parameters
Baselines: STBP-tdBN SEW ResNet MS ResNet TEBN TET Spikformer Spike-driven Transformer OTTT SLTT S-RevSNN LIAF-Net PLIF
Builds on: S-RevSNN OTTT SLTT Spike-driven Transformer ConvNeXt ReZero
Closest papers by meaning
- Memory-Efficient Reversible Spiking Neural Networks (AAAI 2024 · similarity 0.95)
- Temporal Flexibility in Spiking Neural Networks: Towards Generalization Across Time Steps and Deployment Friendliness (ICLR 2025 · similarity 0.91)
- ReverB-SNN: Reversing Bit of the Weight and Activation for Spiking Neural Networks (ICML 2025 · similarity 0.89)
- Are Conventional SNNs Really Efficient? A Perspective from Network Quantization (CVPR 2024 · similarity 0.89)
- Rethinking Spiking Neural Networks from an Ensemble Learning Perspective (ICLR 2025 · similarity 0.88)
- Temporal Misalignment in ANN-SNN Conversion and its Mitigation via Probabilistic Spiking Neurons (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.