Research map / Architectures and efficiency / Spiking networks
Spiking transformers
Adapts transformer architectures to spiking networks with token sparsification and ANN-to-SNN conversion.
87 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2020–2026) · an approach in Spiking networks, keyword cluster “snn · energy · ann”
State History Benchmarks Tried Baselines Checklist Preprints Who Export
Where it stands
Still moving The best reported result is still improving on 1 of 5 tracked benchmarks.
12 papers in 2023–24 and 71 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic grew, from 16.4% to 45.8%.
Open questions the data points to
- ImageNet-1K (top-1 accuracy): best went from 85.65 (up to 2024) to 90.03 in 2025 — TTFSFormer: A TTFS-based Lossless Conversion of Spiking Transformer (ICML 2025)
What looks settled
- CIFAR-10 (accuracy): best up to 2024 97.4, best in 2025–26 97.17 from 7 papers, at most 0.2% better, if at all.
- CIFAR100 (accuracy): best up to 2024 86.3, best in 2025–26 83.06 from 7 papers, at most 0.2% better, if at all.
- CIFAR10-DVS (accuracy): best up to 2024 84.8, best in 2025–26 84.2 from 6 papers, at most 0.2% better, if at all.
- DVS128 Gesture (accuracy): best up to 2024 99.3, best in 2025–26 98.61 from 2 papers, at most 0.2% better, if at all.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2020 · SpikeGrad: An ANN-equivalent Computation Model for Implementing Backpropagation with Spikes (ICLR 2020 · 13 citations)
- 2021 · Self-Supervised Learning of Event-Based Optical Flow with Spiking Neural Networks (NeurIPS 2021 · 60 citations)
- 2022 · Training Spiking Neural Networks with Local Tandem Learning (NeurIPS 2022 · 20 citations)
- 2023 · Spikformer: When Spiking Neural Network Meets Transformer (ICLR 2023 · 104 citations)
- 2024 · SpikingResformer: Bridging ResNet and Vision Transformer in Spiking Neural Networks (CVPR 2024 · 82 citations)
- 2025 · Spiking Transformer with Spatial-Temporal Attention (CVPR 2025 · 25 citations)
- 2026 · Exploring the Potentials of Spiking Neural Networks for Image Deraining (AAAI 2026 · 2 citations)
Earliest papers here
- SpikeGrad: An ANN-equivalent Computation Model for Implementing Backpropagation with Spikes (ICLR 2020 · 13 citations)
- Self-Supervised Learning of Event-Based Optical Flow with Spiking Neural Networks (NeurIPS 2021 · 60 citations)
- Training Spiking Neural Networks with Local Tandem Learning (NeurIPS 2022 · 20 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
CIFAR-10 · accuracy (higher is better)
12 papers, 2023–2025 · flat since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2023 | 95.6 | Spike-driven Transformer (NeurIPS 2023 · 48 citations) |
| 2024 | 97.4 | SpikingResformer: Bridging ResNet and Vision Transformer in Spiking Neural Networks (CVPR 2024 · 82 citations) |
CIFAR100 · accuracy (higher is better)
12 papers, 2023–2025 · flat since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2023 | 78.4 | Spike-driven Transformer (NeurIPS 2023 · 48 citations) |
| 2024 | 86.3 | SpikedAttention: Training-Free and Fully Spike-Driven Transformer-to-SNN Conversion with Winner-Oriented Spike Shift for Softmax Operation (NeurIPS 2024 · 9 citations) |
CIFAR10-DVS · accuracy (higher is better)
10 papers, 2023–2025 · flat since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2023 | 80.9 | Spikformer: When Spiking Neural Network Meets Transformer (ICLR 2023 · 104 citations) |
| 2024 | 84.8 | SpikingResformer: Bridging ResNet and Vision Transformer in Spiking Neural Networks (CVPR 2024 · 82 citations) |
ImageNet-1K · top-1 accuracy (higher is better)
6 papers, 2023–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2023 | 77.07 | Spike-driven Transformer (NeurIPS 2023 · 48 citations) |
| 2024 | 85.65 | QKFormer: Hierarchical Spiking Transformer using Q-K Attention (NeurIPS 2024 · 29 citations) |
| 2025 | 90.03 | TTFSFormer: A TTFS-based Lossless Conversion of Spiking Transformer (ICML 2025) |
DVS128 Gesture · accuracy (higher is better)
5 papers, 2023–2025 · flat since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2023 | 99.3 | Spike-driven Transformer (NeurIPS 2023 · 48 citations) |
What has been tried
30 papers here report 347 measured changes (a component added, removed or swapped, compared with a baseline): helped 57%, no clear effect 10%, hurt 30%, mixed 3%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| add the multi-stage architecture | ImageNet100 · Accuracy | 85.32 -> 88.06 | Helped | SpikingResformer: Bridging ResNet and Vision Transformer in Spiking Neural Networks (CVPR 2024 · 82 citations) |
| add SPEDS | CIFAR100 · accuracy | 78.21% -> 80.26%; +2.05% | Helped | QKFormer: Hierarchical Spiking Transformer using Q-K Attention (NeurIPS 2024 · 29 citations) |
| Add timestep shrinkage | CIFAR10-DVS, N-Caltech101, DVS-Gesture, average timestep 5 · accuracy | 67.08 -> 71.53; 66.45 -> 75.42; 80.29 -> 85.42 | Helped | Shrinking Your TimeStep: Towards Low-Latency Neuromorphic Object Recognition with Spiking Neural Networks (AAAI 2024 · 28 citations) |
| Use spatial-temporal attention instead of temporal-only attention | CIFAR100 · accuracy | 77.7% -> 79.9% | Helped | Spiking Transformer with Spatial-Temporal Attention (CVPR 2025 · 25 citations) |
| Remove SepConv | ImageNet-1K · top-1 accuracy | 75.4 -> 75.1 | Hurt | Spike-driven Transformer V2: Meta Spiking Neural Network Architecture Inspiring the Design of Next-generation Neuromorphic Chips (ICLR 2024 · 19 citations) |
| Use vanilla mask embedding instead of SDME | ADE20k · mIoU | 46.3 -> 41.8 | Hurt | Spike2Former: Efficient Spiking Transformer for High-performance Image Segmentation (AAAI 2025 · 13 citations) |
| Add GC | CIFAR-100 · accuracy | 72.30% -> 73.22% | Helped | STAA-SNN: Spatial-Temporal Attention Aggregator for Spiking Neural Networks (CVPR 2025 · 13 citations) |
| Increase the number of encoder blocks from 8 to 10 | ImageNet-1K · accuracy | 76.28 -> 78.66 | Helped | Spiking Transformer: Introducing Accurate Addition-Only Spiking Self-Attention for Transformer (CVPR 2025 · 12 citations) |
| Convert Swin-T without inserting ReLUs at every layer | ImageNet classification · accuracy | 77.4 -> 80.0 | Helped | SpikedAttention: Training-Free and Fully Spike-Driven Transformer-to-SNN Conversion with Winner-Oriented Spike Shift for Softmax Operation (NeurIPS 2024 · 9 citations) |
| Adaptive-Firing Neuron Model (AdaFire) added to calibration | ImageNet ablation · Accuracy (%) | 56.74 -> 68.45 (+11.71) | Helped | Adaptive Calibration: A Unified Conversion Framework of Spiking Neural Networks (AAAI 2025 · 6 citations) |
| Add the α-SSA attention-score linear transformation while retaining dot-product similarity (α = 0) | CIFAR100 · accuracy | 78.21% to 79.04% | Helped | Rethinking Spiking Self-Attention Mechanism: Implementing a-XNOR Similarity Calculation in Spiking Transformers (CVPR 2025 · 6 citations) |
| Add HDA in place of separate TA and CA attention | AID-tiny · PSNR (dB) | +0.12 dB | Helped | Spiking Meets Attention: Efficient Remote Sensing Image Super-Resolution with Attention Spiking Neural Networks (NeurIPS 2025 · 5 citations) |
Baseline numbers that disagree
The same baseline on the same benchmark and metric, reported more than 5% apart by different papers here. Training budgets, data splits and reimplementations differ, so state which number you compare with.
| Baseline | Benchmark | Metric | Reported | Papers |
|---|---|---|---|---|
| Baseline | CIFAR-100 | accuracy | 73.17 – 76.95 | 2 papers, e.g. NeurIPS 2023 and ICLR 2025 |
What reviewers will expect
Datasets and baselines named in the 31 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- CIFAR-10 · 11 of 31 papers
- CIFAR-100 · 11 of 31 papers
- CIFAR10-DVS · 8 of 31 papers
- ImageNet-1K · 8 of 31 papers
- ImageNet · 4 of 31 papers
- ADE20K · 3 of 31 papers
- N-Caltech101 · 2 of 31 papers
- PASCAL VOC 2012 · 2 of 31 papers
Baselines
- Spikformer · 9 of 31 papers
- Spikingformer · 6 of 31 papers
- Spike-driven Transformer · 6 of 31 papers
- PLIF · 4 of 31 papers
- TET · 4 of 31 papers
- QKFormer · 4 of 31 papers
- DSR · 3 of 31 papers
- SpikingResformer · 3 of 31 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 83 papers; from OpenAlex and the papers' affiliations).
- Peking University · 10 papers
- Chinese Academy of Sciences · 9 papers
- University of Electronic Science and Technology of China · 6 papers
- Liaoning Technical University · 3 papers
- Peng Cheng Laboratory · 3 papers
- National University of Singapore · 3 papers
- Zhejiang University · 3 papers
- Harbin Institute of Technology · 3 papers
- Hong Kong University of Science and Technology · 2 papers
- Northwestern University · 2 papers
Use it
Download BibTeX of 25 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Training deep spiking networks (141 papers)
- Event-camera spiking networks (13 papers)
Elsewhere in the topic
- Sparse efficient spiking sequence models (26 papers)
- Bio-inspired spiking neuron models (22 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).