Research map / Architectures and efficiency / Spiking networks
STAA-SNN: Spatial-Temporal Attention Aggregator for Spiking Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes STAA-SNN, a plug-in spatial-temporal attention aggregator (global context blocks, learnable position encoding, step attention) plus time-step random dropout for directly trained spiking neural networks, reporting SOTA accuracy on CIFAR-10/100, ImageNet and CIFAR10-DVS.
Problem
Directly trained spiking neural networks still lag behind ANNs in accuracy, and existing SNN attention and temporal aggregation (simple addition of inputs across timesteps) lose information and do not model temporal dependencies well.
Approach
Integrates a global-context self-attention block, a learnable position encoding applied before aggregation, and a step attention module that reweights features per timestep into the LIF neuron update, plus a time-step random dropout training strategy that randomly replaces the augmentation module with plain addition at timesteps.
Key results
- STAA-SNN reaches 97.14% on CIFAR-10 (ResNet-19, 4 timesteps), 82.05% on CIFAR-100 and 70.40% on ImageNet (ResNet-34, 4 timesteps).
- On CIFAR10-DVS, VGG-13 with 16 timesteps achieves 82.10%, and DVS128 Gesture reaches 98.61%.
- Ablation on CIFAR-100 (ResNet-20, T=4) shows cumulative gains of +0.92 (GC), +1.14 (SA), and +0.82 (PE and TSRD combined), totaling +2.80% over baseline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use STAA-SNN instead of MPBN | ResNet-19, 4 timesteps · CIFAR-10 | accuracy: 96.52% -> 97.14%± 0.10% | Helped |
| Use STAA-SNN instead of MPBN | ResNet-19, 4 timesteps · CIFAR-100 | accuracy: 80.1% -> 82.05%± 0.10% | Helped |
| Use STAA-SNN instead of Real Spike | ResNet-34, 4 timesteps · ImageNet | accuracy: 67.69% -> 70.40%± 0.15% | Helped |
| Use STAA-SNN instead of Spikeformer | 16-timestep event classification · CIFAR10-DVS | accuracy: 80.9% -> 82.10%± 0.20% | Helped |
| Use STAA-SNN instead of Spike-driven Transformer | 16-timestep event classification · DVS128 Gesture | accuracy: 99.30% -> 98.61%± 0.20% | Hurt |
| Add GC (ablation) | Baseline SNN with ResNet-20 · CIFAR-100 | accuracy: 72.30% -> 73.22% | Helped |
| Add PE to GC (ablation) | GC-equipped SNN with ResNet-20 · CIFAR-100 | accuracy: 73.22% -> 73.79% | Helped |
| Add TSRD to GC + PE (ablation) | GC + PE SNN with ResNet-20 · CIFAR-100 | accuracy: 73.79% -> 73.96% | Helped |
| Add SA to GC + PE (ablation) | GC + PE SNN with ResNet-20 · CIFAR-100 | accuracy: 73.79% -> 74.78% | Helped |
| Add TSRD to GC + PE + SA (ablation) | STAA-SNN with ResNet-20 · CIFAR-100 | accuracy: 74.78% -> 75.10% | Helped |
| Place PE at the pre-aggregation spatial input port (ablation) | SNN · CIFAR-10 | accuracy: 94.08% -> 94.29% | Helped |
| Use learnable rather than fixed PE (ablation) | SNN · CIFAR-10 | accuracy: 94.42% -> 94.99% | Helped |
Surprising findings
- On DVS128 Gesture, the Spike-driven Transformer (99.30%) outperforms the proposed method (98.61%), which the table does not highlight as SOTA.
Limitations
- Energy consumption of the method remains a concern, as noted by the authors
- Mixed-results on DVS128 Gesture where a prior transformer method performs better
- Position encoding placement and dropout probability chosen by ablation on CIFAR-10/100, with limited analysis of generalization beyond convolutional architectures
Experimental setup
Datasets: CIFAR-10 CIFAR-100 ImageNet CIFAR10-DVS DVS128 Gesture
Benchmarks: CIFAR-10 CIFAR-100 ImageNet CIFAR10-DVS DVS128 Gesture
Metrics: top-1 accuracy
Baselines: TET GLIF MPBN IM-LIF Spikformer Spike-driven Transformer IM-Loss LSG PFA MS-ResNet
Builds on: LIF neuron STBP Global Context block Squeeze-and-excitation networks MPBN
Closest papers by meaning
- FSTA-SNN:Frequency-Based Spatial-Temporal Attention Module for Spiking Neural Networks (AAAI 2025 · similarity 0.92)
- Advancing Spiking Neural Networks Towards Multiscale Spatiotemporal Interaction Learning (AAAI 2025 · similarity 0.91)
- Unveiling the Spatial-temporal Effective Receptive Fields of Spiking Neural Networks (NeurIPS 2025 · similarity 0.89)
- Spiking Vision Transformer with Saccadic Attention (ICLR 2025 · similarity 0.89)
- TS-SNN: Temporal Shift Module for Spiking Neural Networks (ICML 2025 · similarity 0.89)
- Spiking Transformer with Spatial-Temporal Attention (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.