Research map / Architectures and efficiency / Spiking networks
Spikformer: When Spiking Neural Network Meets Transformer
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Spikformer-8-768 instead of SEW-ResNet-152 | ImageNet classification · ImageNet val | top-1 accuracy: 69.26 -> 74.81 | Helped |
| Use SSA rather than softmax attention (ablation) | Spikformer · CIFAR10 | accuracy: 94.97 -> 95.19 | Helped |
| Use SSA rather than softmax attention (ablation) | Spikformer · CIFAR100 | accuracy: 77.92 -> 77.86 | Hurt |
| Use SSA rather than softmax attention (ablation) | Spikformer · CIFAR10-DVS | accuracy: 80.00 -> 80.90 | Helped |
| Replace SSA with vanilla attention using spike-form V (ablation) | Spikformer-8-512 · ImageNet val | top-1 accuracy: 72.70 -> 73.38 | Hurt |
| Use float-point V in vanilla attention instead of spike-form V in SSA (ablation) | Spikformer-8-512 · ImageNet val | top-1 accuracy: 73.38 -> 73.96 | Helped |
| Reduce the embedding dimension from 384 to 256 (ablation) | Spikformer-4 on CIFAR10 · CIFAR10 | accuracy: 93.94 -> 95.19 | Hurt |
| Reduce encoder depth from 4 blocks to 2 (ablation) | Spikformer-384 on CIFAR10 · CIFAR10 | accuracy: 94.80 -> 95.19 | Hurt |
| Extend training to 400 epochs (ablation) | Spikformer-4-384 · CIFAR10 | accuracy: 95.19 -> 95.51 | Helped |
| Extend training to 400 epochs (ablation) | Spikformer-4-384 · CIFAR100 | accuracy: 77.86 -> 78.21 | Helped |
| Increase simulation time steps from 1 to 4 (ablation) | Spikformer-4-384 · CIFAR10 | accuracy: 93.51 -> 95.19 | Helped |
| Add 1D max-pooling after Q and K (ablation) | Spikformer on DVS128 Gesture · DVS128 Gesture | accuracy: 97.9% -> 98.3% | Helped |
Closest papers by meaning
- Spiking Transformer: Introducing Accurate Addition-Only Spiking Self-Attention for Transformer (CVPR 2025 · similarity 0.93)
- SpikingResformer: Bridging ResNet and Vision Transformer in Spiking Neural Networks (CVPR 2024 · similarity 0.93)
- Spiking Vision Transformer with Saccadic Attention (ICLR 2025 · similarity 0.91)
- Spike-driven Transformer V2: Meta Spiking Neural Network Architecture Inspiring the Design of Next-generation Neuromorphic Chips (ICLR 2024 · similarity 0.90)
- SpikF: Spiking Fourier Network for Efficient Long-term Prediction (ICML 2025 · similarity 0.90)
- STAA-SNN: Spatial-Temporal Attention Aggregator for Spiking Neural Networks (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.