Research map / Architectures and efficiency / Spiking networks
Towards Lossless Memory-efficient Training of Spiking Neural Networks via Gradient Checkpointing and Spike Compression
Yifan Huang, Wei Fang, Zecheng Hao, Zhengyu Ma, Yonghong Tian
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace SpikingJelly CuPy LIF with Triton MELIF kernel (ablation) | SNN BPTT training · CIFAR10-DVS | peak allocated memory (MB): 6131.07 -> 4865.06 | Improved |
| replace SpikingJelly CuPy LIF with Triton MELIF kernel (ablation) | SNN BPTT training · CIFAR10-DVS | throughput (samples/s): 290.26 -> 331.30 | Improved |
| add layer-wise gradient checkpointing and spike compression (O1) (ablation) | MELIF Spiking VGG · CIFAR10-DVS | peak allocated memory (MB): 4865.06 -> 2887.75 | Improved |
| add layer-wise gradient checkpointing and spike compression (O1) (ablation) | MELIF Spiking VGG · CIFAR10-DVS | throughput (samples/s): 331.30 -> 246.81 | Hurt |
| add spatio-temporal segment partitioning (O3) (ablation) | MELIF + O1 · CIFAR10-DVS | peak allocated memory (MB): 2887.75 -> 2349.39 | Improved |
| add greedy segment restoration (O4) (ablation) | MELIF + O3 · CIFAR10-DVS | throughput (samples/s): 247.83 -> 270.79 | Improved |
| add spike compression at O1 (ablation) | MELIF + GC Spiking VGG · CIFAR10-DVS | peak allocated memory (MB): 2892.63 -> 2887.75 | Improved |
| add spike compression at O3 (ablation) | MELIF + GC + partitioning · CIFAR10-DVS | peak allocated memory (MB): 2530.66 -> 2349.39 | Improved |
| add AMP | MELIF + O4 QKFormer · ImageNet 1/32 subset | peak allocated memory (MB): 5219.93 -> 3158.02 | Improved |
| add LOMO | MELIF + O4 QKFormer · ImageNet 1/32 subset | peak allocated memory (MB): 5219.93 -> 5190.60 | No clear effect |
| add LOMO on top of AMP | MELIF + O4 QKFormer · ImageNet 1/32 subset | peak allocated memory (MB): 3158.02 -> 3142.86 | No clear effect |
| apply O4 pipeline to parallel spiking neurons | SEW ResNet-34 with PSN · ImageNet 1/32 subset | peak allocated memory (MB): 7602.64 -> 2544.28 | Improved |
Closest papers by meaning
- Advancing Training Efficiency of Deep Spiking Neural Networks through Rate-based Backpropagation (NeurIPS 2024 · similarity 0.91)
- Memory-Efficient Reversible Spiking Neural Networks (AAAI 2024 · similarity 0.90)
- Enhanced Self-Distillation Framework for Efficient Spiking Neural Network Training (NeurIPS 2025 · similarity 0.90)
- Take A Shortcut Back: Mitigating the Gradient Vanishing for Training Spiking Neural Networks (NeurIPS 2024 · similarity 0.90)
- Temporal Flexibility in Spiking Neural Networks: Towards Generalization Across Time Steps and Deployment Friendliness (ICLR 2025 · similarity 0.89)
- High-Performance Temporal Reversible Spiking Neural Networks with $\mathcal{O}(L)$ Training Memory and $\mathcal{O}(1)$ Inference Cost (ICML 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
