Research map / Architectures and efficiency / Spiking networks
Rethinking the Membrane Dynamics and Optimization Objectives of Spiking Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a learnable initial membrane potential (IMP), last-time-step (LTS) output for static tasks, and a label-smoothed TET loss (TET-S) to improve directly trained SNN accuracy on static and neuromorphic benchmarks.
Problem
Directly trained spiking neural networks have limited firing patterns within few time steps because the initial membrane potential is fixed to 0, and existing losses (TET, SDT) converge slowly or conflict with regularization on static and neuromorphic tasks.
Approach
Treat the initial membrane potential of LIF/IF neurons as a learnable per-neuron parameter (uniformly initialized around zero) to enlarge spike pattern sets and mappings; use only the last time step output for static tasks (LTS); and replace TET's MSE firing-rate regularizer with label smoothing (TET-S).
Key results
- On CIFAR10-DVS, IMP+TET-S reaches 87.80% at 8 time steps, and 87.86% on N-Caltech101 at resolution 128.
- On ImageNet1k, IMP+LTS with SEW-ResNet50 reaches 71.83% (+4.05%), surpassing vanilla SEW-ResNet152 (69.26%) with 4 time steps.
- IMP adds almost no execution time overhead (about ±1.03%) relative to vanilla IF neurons.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add learnable IMP and label-smoothed TET loss | VGG · CIFAR10-DVS, resolution 48 | accuracy: 85.90 -> 87.10 | Helped |
| Add learnable IMP and label-smoothed TET loss | VGG · N-Caltech101, resolution 128 | accuracy: 85.91 -> 87.86 | Helped |
| Apply LTS to SEW-ResNet50 (ablation) | SEW-ResNet50 · ImageNet1k | accuracy: 67.78 -> 71.24 | Helped |
| Add learnable IMP to SEW-ResNet50 with LTS (ablation) | SEW-ResNet50 with LTS · ImageNet1k | accuracy: 71.24 -> 71.83 | Helped |
| Add IMP to LTS (ablation) | SEW-ResNet18 · ImageNet100 | accuracy: 80.20 -> 80.80 | Helped |
| Replace TET with LTS (ablation) | SEW-ResNet18 · ImageNet100 | accuracy: 78.50 -> 80.20 | Helped |
| Add label smoothing with ϵ = 0.01 to TET (ablation) | VGG · CIFAR10-DVS | accuracy: 84.90 -> 86.10 | Helped |
| Vary label smoothing from ϵ = 0.01 to ϵ = 0.1 (ablation) | TET-S on VGG · CIFAR10-DVS | accuracy: 86.10 -> 85.60 | Hurt |
| Vary label smoothing from ϵ = 0.01 to ϵ = 0.001 (ablation) | TET-S on VGG · CIFAR10-DVS | accuracy: 86.10 -> 85.40 | Hurt |
| Add IMP to TET-S with λ = 0.2 (ablation) | VGG · CIFAR10-DVS | accuracy: 86.10 -> 87.10 | Helped |
| Increase IMP initialization bound from λ = 0.2 to λ = 0.4 (ablation) | IMP+TET-S on VGG · CIFAR10-DVS | accuracy: 87.10 -> 86.40 | Hurt |
| Use LTS instead of TET output on VGG (ablation) | VGG · CIFAR10-DVS, 10 time steps | accuracy: 85.8 -> 82.9 | Hurt |
Surprising findings
- Using TET loss on static tasks converges more slowly than SDT, and LTS (supervising only the last time step) can lose information on DVS tasks with many time steps.
Limitations
- IMP is sensitive to initialization due to small gradients, and learnable IMP adds one parameter per neuron.
- LTS benefit diminishes at large time steps (recommended only below 8), and it hurts on DVS tasks.
- No unified loss found that works well on both static and neuromorphic tasks.
Experimental setup
Datasets: CIFAR10 CIFAR100 ImageNet1k ImageNet100 CIFAR10-DVS DVS Gesture N-Caltech101
Benchmarks: CIFAR10 CIFAR100 ImageNet1k ImageNet100 CIFAR10-DVS N-Caltech101
Metrics: top-1 accuracy execution time SOPs power (mJ)
Baselines: TET SDT PSN TEBN GLIF NDA Spikformer EventRPG TKS STR
Builds on: temporal efficient training standard direct training SEW ResNet tdBN parametric LIF
Closest papers by meaning
- Enhanced Self-Distillation Framework for Efficient Spiking Neural Network Training (NeurIPS 2025 · similarity 0.92)
- Rethinking Spiking Neural Networks from an Ensemble Learning Perspective (ICLR 2025 · similarity 0.92)
- EnOF-SNN: Training Accurate Spiking Neural Networks via Enhancing the Output Feature (NeurIPS 2024 · similarity 0.90)
- Adaptive Calibration: A Unified Conversion Framework of Spiking Neural Networks (AAAI 2025 · similarity 0.90)
- Robust Stable Spiking Neural Networks (ICML 2024 · similarity 0.90)
- Take A Shortcut Back: Mitigating the Gradient Vanishing for Training Spiking Neural Networks (NeurIPS 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.