Research map / Architectures and efficiency / Spiking networks
Advancing Spiking Neural Networks Towards Multiscale Spatiotemporal Interaction Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a Spiking Multiscale Attention (SMA) module and an Attention Zoneout (AZO) regularizer for SNNs that capture multiscale spatiotemporal interactions, reaching SOTA on several neuromorphic datasets and 77.1% on ImageNet-1K with a 104-layer SNN ResNet.
Problem
SNNs often ignore multiscale information and spatiotemporal correlations in event data, treating each frame as a static image, which contributes to the performance gap with ANNs.
Approach
Extend the SE attention module to multiple scales and compute time- and channel-wise attention weights from spatiotemporal correlations (SMA); then use those attention weights to choose where to replace hidden units with previous-timestep values as noise for pseudo-ensemble regularization (AZO), instead of random Zoneout.
Key results
- SMA-AZO-VGG reaches 84.0% on CIFAR10-DVS and 84.6% on N-Caltech101, with SMA-VGG at 83.7% on N-Caltech101 (vs. TCJA-SNN 78.5%).
- SMA-AZO-MS-ResNet-104 reaches 77.05% top-1 on ImageNet-1K at 224 (288 input crop) with 5 timesteps, versus MS-ResNet-104* at 76.02%.
- SMA-VGG reaches 98.3% on DVS128 Gesture, SMA-AZO-VGG 98.6%, matching non-transformer methods at T=16.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add SMA to VGG | VGG · CIFAR10-DVS | accuracy | Helped |
| Add SMA to VGG | VGG · N-Caltech101 | accuracy | Helped |
| Add AZO regularization to SMA-VGG | SMA-VGG · DVS128 Gesture | accuracy: 98.3 -> 98.6 | Helped |
| Add AZO regularization to SMA-VGG | SMA-VGG · CIFAR10-DVS | accuracy: 83.1 -> 84.0 | Helped |
| Add AZO regularization to SMA-VGG | SMA-VGG · N-Caltech101 | accuracy: 83.7 -> 84.6 | Helped |
| Add SMA to MS-ResNet-18 | MS-ResNet-18 · ImageNet-1K | Top-1 accuracy: +5.36% | Helped |
| Add SMA to MS-ResNet-34 | MS-ResNet-34 · ImageNet-1K | Top-1 accuracy: +0.77% | Helped |
| Add SMA and AZO to MS-ResNet-104 | MS-ResNet-104 · ImageNet-1K | Top-1 accuracy: +1.03% | Helped |
| Use scale 4 rather than no SMA (ablation) | SMA · DVS128 Gesture | accuracy: 96.52 -> 97.91(±0.34) | Helped |
| Use scale 2 rather than no SMA (ablation) | SMA · DVS128 Gesture | accuracy: 96.52 -> 96.52(±0.34) | No clear effect |
| Use scale 3 rather than no SMA (ablation) | SMA · DVS128 Gesture | accuracy: 96.52 -> 97.21(±0.69) | Helped |
| Use scale 5 rather than no SMA (ablation) | SMA · DVS128 Gesture | accuracy: 96.52 -> 97.73(±0.17) | Helped |
Surprising findings
- Placing SMA after BatchNorm (L2) caused the network to fail to converge, attributed to poor attention extraction from normalized data.
- The LIF-neuron version of SMA performs only marginally worse than the ReLU version, with identical accuracy on DVS128 Gesture.
Limitations
- Evaluation on static ImageNet uses spiking coding, so spatiotemporal interaction information is limited there, as the authors acknowledge.
- AZO hyperparameters (RCR, RTR) must be tuned per dataset, and N-Caltech101 was not exhaustively tuned.
- Inference overhead increases with SMA and with scale count (e.g., 6 scales raise inference time to 42.43s from 18.98s without SMA).
Experimental setup
Datasets: DVS128 Gesture CIFAR10-DVS N-Caltech101 ImageNet-1K
Benchmarks: DVS128 Gesture CIFAR10-DVS N-Caltech101 ImageNet-1K
Metrics: top-1 accuracy inference time spike firing rate
Baselines: PLIF Spikformer tdBN SEW-ResNet TA-SNN TCJA-SNN Spike-driven Transformer MA-ResNet MS-ResNet Meta-SpikeFormer
Builds on: Squeeze-and-Excitation Zoneout MS-ResNet TCJA-SNN Multi-dimensional attention SNN
Closest papers by meaning
- STAA-SNN: Spatial-Temporal Attention Aggregator for Spiking Neural Networks (CVPR 2025 · similarity 0.91)
- FSTA-SNN:Frequency-Based Spatial-Temporal Attention Module for Spiking Neural Networks (AAAI 2025 · similarity 0.89)
- Rethinking Spiking Neural Networks from an Ensemble Learning Perspective (ICLR 2025 · similarity 0.87)
- An Efficient Knowledge Transfer Strategy for Spiking Neural Networks from Static to Event Domain (AAAI 2024 · similarity 0.87)
- Temporal Flexibility in Spiking Neural Networks: Towards Generalization Across Time Steps and Deployment Friendliness (ICLR 2025 · similarity 0.87)
- Spatio-Temporal Interactive Learning for Efficient Image Reconstruction of Spiking Cameras (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.