Research map / Architectures and efficiency / Spiking networks
SpikingResformer: Bridging ResNet and Vision Transformer in Spiking Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Dual Spike Self-Attention (DSSA), a fully spike-driven self-attention with a derived scaling method, and SpikingResformer, a ResNet-style multi-stage spiking vision transformer that reaches 79.40% ImageNet top-1 with 4 time-steps.
Problem
Spiking vision transformers rely on shallow convolutional tokenizers for local features and their spiking self-attention lacks proper scaling, so they cannot handle large multi-scale feature maps or be deployed in multi-stage architectures.
Approach
Dual Spike Transformation computes attention from AND-gated dual spike inputs (only synaptic accumulations), with a theoretically derived scaling factor based on firing rates that makes it trainable at multiple scales; this is combined with a ResNet-style multi-stage backbone and group-wise convolution in the feed-forward network.
Key results
- SpikingResformer-L reaches 79.40% top-1 on ImageNet at 288x288 inference with 4 time-steps, the reported SNN state of the art.
- SpikingResformer-Ti achieves 74.34% top-1 with 11.14M parameters and 2.46 mJ, outperforming Spike-driven Transformer-8-384 by 2.06% with fewer parameters.
- On ImageNet100 ablation, removing the scaling factor or using SSA/SDSA-style attention causes training non-convergence, while removing the multi-stage architecture costs 2.74% accuracy.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use SpikingResformer-L instead of Spike-driven Transformer-8-768 | ImageNet classification · ImageNet val | Top-1 accuracy: 76.32 -> 78.77 | Helped |
| use transfer learning with SpikingResformer instead of transfer learning with Spikformer | CIFAR10 · CIFAR10 | Accuracy: 97.03 -> 97.40 | Helped |
| use transfer learning with SpikingResformer instead of transfer learning with Spikformer | CIFAR100 · CIFAR100 | Accuracy: 83.83 -> 85.98 | Helped |
| transfer from ImageNet pretraining rather than direct training | SpikingResformer · CIFAR10-DVS | Accuracy: 81.3 -> 84.8 | Helped |
| transfer from ImageNet pretraining rather than direct training | SpikingResformer · DVSGesture | Accuracy: 99.3 -> 93.4 | Hurt |
| add the multi-stage architecture (ablation) | SpikingResformer-S · ImageNet100 | Accuracy: 85.32 -> 88.06 | Helped |
| add the group-wise convolution layer in GWSFFN (ablation) | SpikingResformer-S · ImageNet100 | Accuracy: 84.64 -> 88.06 | Helped |
| replace DSSA with SSA (ablation) | SpikingResformer-S · ImageNet100 | Accuracy | Hurt |
| replace DSSA with SDSA (ablation) | SpikingResformer-S · ImageNet100 | Accuracy | Hurt |
| replace p×p convolutions in DST with 1×1 convolutions (ablation) | SpikingResformer-S · ImageNet100 | Accuracy: 88.06 -> 86.14 | Hurt |
| remove the DSSA scaling factors (ablation) | SpikingResformer-S · ImageNet100 | Accuracy | Hurt |
| replace the proposed DSSA scaling factors with 1/√d (ablation) | SpikingResformer-S · ImageNet100 | Accuracy | Hurt |
Surprising findings
- Existing spiking self-attention variants (SSA and SDSA) fail to converge when applied to the multi-stage, multi-scale setting, and removing the proposed scaling factor also prevents convergence.
- Transfer from ImageNet pre-trained models underperforms direct training on DVSGesture (93.4% vs 99.3% for Spike-driven Transformer), attributed to temporal content differences.
Limitations
- Transfer learning to DVSGesture underperforms direct training, indicating poor transfer from static pretraining to temporally rich neuromorphic data.
- Energy figures are estimates from synaptic operation counts rather than measured on neuromorphic hardware.
Experimental setup
Datasets: ImageNet ImageNet100 CIFAR10 CIFAR100 CIFAR10-DVS DVSGesture
Benchmarks: ImageNet CIFAR10 CIFAR100 CIFAR10-DVS DVSGesture
Metrics: top-1 accuracy parameters SOPs energy (mJ)
Baselines: Spikformer Spikingformer Spike-driven Transformer SEW ResNet STBP-tdBN TET PLIF Dspike
Builds on: Spikformer Spike-driven Transformer SEW ResNet Spikingformer Vision Transformer
Closest papers by meaning
- Spiking Transformer: Introducing Accurate Addition-Only Spiking Self-Attention for Transformer (CVPR 2025 · similarity 0.92)
- Spiking Vision Transformer with Saccadic Attention (ICLR 2025 · similarity 0.90)
- STEP: A Unified Spiking Transformer Evaluation Platform for Fair and Reproducible Benchmarking (NeurIPS 2025 · similarity 0.89)
- Spike-driven Transformer V2: Meta Spiking Neural Network Architecture Inspiring the Design of Next-generation Neuromorphic Chips (ICLR 2024 · similarity 0.89)
- Decision SpikeFormer: Spike-Driven Transformer for Decision Making (CVPR 2025 · similarity 0.88)
- Spiking Transformer with Spatial-Temporal Attention (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.