Research map / Architectures and efficiency / Spiking networks
Surrogate Module Learning: Reduce the Gradient Error Accumulation in Training Spiking Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add surrogate modules with self-distillation (full SML) vs end-to-end training (ablation) | spiking ResNet-18 (T=2) · CIFAR-100 ablation | accuracy: +3.79% over end-to-end | Helped |
| use one-direction distillation (BYOT-style) instead of two-direction surrogate learning (ablation) | spiking ResNet-18 (T=2) · CIFAR-100 ablation | accuracy: -2.59% vs full SML | Hurt |
| remove backbone gradient flow (local learning) vs full SML (ablation) | spiking ResNet-18 (T=2) · CIFAR-100 ablation | accuracy: -2.06% for {1,1,1} vs {0,1,1}; local only 68.8 | Hurt |
| add auxiliary network without distillation (plain local learning) (ablation) | spiking ResNet-18 (T=2) · CIFAR-100 ablation | accuracy: 68.8 vs baseline | No clear effect |
| add one surrogate module to ANN (AdamW) (ablation) | ANN ResNet-18 · CIFAR-10/CIFAR-100 | accuracy: -0.35% CIFAR-10, -0.31% CIFAR-100 | Hurt |
| surrogate module learning on ImageNet ResNet-34 vs training from scratch | spiking ResNet-34 · ImageNet | accuracy: +3.46% | Helped |
| surrogate module learning on ES-ImageNet | spiking ResNet-18 (T=8) · ES-ImageNet | accuracy: +4.87% vs prior ResNet-18 work | Helped |
| surrogate module learning on CIFAR-10 static (ResNet-18 vs IM-Loss ResNet-19) | spiking ResNet-18 · CIFAR-10 | accuracy: +0.05% vs IM-Loss at T=6 by text, but 95.12 is below IM-Loss 95.49 as printed in table | Mixed |
| add TET loss on top of surrogate module learning (ablation) | spiking ResNet-18 · DVS-CIFAR10 | accuracy: 83.19 -> 85.23 | Helped |
| surrogate module learning adds training overhead | spiking ResNet-18 (2 surrogate modules) · training cost | training time overhead: +11.88% at T=4 (+53.84% at T=1) | No clear effect |
Closest papers by meaning
- Take A Shortcut Back: Mitigating the Gradient Vanishing for Training Spiking Neural Networks (NeurIPS 2024 · similarity 0.90)
- Adaptive Surrogate Gradients for Sequential Reinforcement Learning in Spiking Neural Networks (NeurIPS 2025 · similarity 0.89)
- A generalized neural tangent kernel for surrogate gradient learning (NeurIPS 2024 · similarity 0.89)
- Enhanced Self-Distillation Framework for Efficient Spiking Neural Network Training (NeurIPS 2025 · similarity 0.89)
- Advancing Training Efficiency of Deep Spiking Neural Networks through Rate-based Backpropagation (NeurIPS 2024 · similarity 0.88)
- EnOF-SNN: Training Accurate Spiking Neural Networks via Enhancing the Output Feature (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.