Research map / Architectures and efficiency / Spiking networks
Spiking Token Mixer: An event-driven friendly Former structure for spiking neural networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes STMixer, an SNN architecture built only from operations supported by event-driven asynchronous hardware (conv, FC, residual paths), replacing spike-spike matrix multiplication with a learned token-mixing weight and removing maxpool, and reports strong accuracy at T=1 on CIFAR and ImageNet.
Problem
Spikformer-style SNNs use spike-by-spike matrix multiplications and max pooling, which break under spike arrival-time jitter on event-driven asynchronous neuromorphic hardware, while clock-driven hardware needs short time steps that hurt accuracy.
Approach
Replace the spiking self-attention (Q K^T V) with a learnable token-mixing weight matrix applied to the value matrix (spike-by-fixed-weight), use multi-head token mixing, and redesign the patch splitting module (IPSPS) without maxpool and with an auxiliary convolutional information path; also use the STM encoder inside surrogate module learning.
Key results
- STMixer-8-768-16 reaches 76.68% top-1 on ImageNet at T=1 with 4.45 mJ estimated energy, above Spikingformer-8-768 (75.85% at T=4, 13.68 mJ).
- STMixer-4-384-32 reaches 80.00% on CIFAR-100 at T=1 (81.87% at T=4), above Spikingformer-4-384 (79.09%).
- On CIFAR10-DVS under a fully event-driven simulator, STMixer(T=25) keeps 50.70% while Spikformer(T=25) drops to 16.60%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use STMixer instead of Spikingformer | SNN on ImageNet · ImageNet | accuracy: 75.85 -> 76.68 | Helped |
| use STMixer instead of Spikingformer | SNN on CIFAR10-DVS · CIFAR10-DVS | accuracy: 79.9 -> 82.67 | Helped |
| use STMixer instead of TA-SNN | SNN on DVS128 Gesture · DVS128 Gesture | accuracy: 98.6 -> 97.19 | Hurt |
| deploy STMixer rather than Spikformer in event-driven mode (T=25) | CIFAR10-DVS event-driven inference · CIFAR10-DVS, event-driven | accuracy: 16.60 -> 50.70 | Helped |
| deploy STMixer rather than Spikformer in event-driven mode (T=40) | CIFAR10-DVS event-driven inference · CIFAR10-DVS, event-driven | accuracy: 10.00 -> 54.10 | Helped |
| add IPSPS instead of using SPS (ablation) | STMixer · CIFAR-100 | accuracy: 78.97 -> 79.55 | Helped |
| use STM instead of SSA (ablation) | STMixer · CIFAR-100 | accuracy: 78.33 -> 79.55 | Helped |
| use STM in the surrogate module instead of a convolutional encoder (ablation) | SML training for STMixer · CIFAR-100 | accuracy: 79.07 -> 79.55 | Helped |
| use SML instead of SDT (ablation) | STMixer training · CIFAR-100 | accuracy: 77.17 -> 79.55 | Helped |
| increase STM head count from 1 to 32 (ablation) | STMixer-4-384 · CIFAR-100 | accuracy: 79.02 -> 80.15 | Helped |
| increase the time step from T=1 to T=4 | STMixer-4-384-32 · CIFAR-100 | accuracy: 80.00±0.21 -> 81.87±0.16 | Helped |
| increase the time step from T=1 to T=4 | STMixer-4-384-32 · CIFAR-10 | accuracy: 95.49±0.13 -> 96.01±0.11 | Helped |
Surprising findings
- Spikformer collapses to chance (10.00%) in the fully event-driven setting at T=40 when bias is removed, while STMixer stays much higher.
- Increasing the number of token-mixing heads from 1 to 32 steadily improves CIFAR-100 accuracy (79.02% to 80.15%).
Limitations
- Energy estimates use an incomplete model that ignores memory overhead, so the energy comparison is approximate.
- Asynchronous chips like Speck support only small networks (e.g., 9 layers), so large STMixer models were only tested in a simulator.
- Event-driven experiments required a two-phase bias-removal pipeline that slightly degrades accuracy.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 ImageNet CIFAR10-DVS DVS128 Gesture
Benchmarks: CIFAR-10 CIFAR-100 ImageNet CIFAR10-DVS DVS128 Gesture
Metrics: top-1 accuracy parameter count FLOPs estimated energy (mJ)
Baselines: Spikformer Spikingformer SML TET tdBN SEW-ResNet SpikeDHSD
Builds on: Spikformer Spikingformer MetaFormer Surrogate module learning SEW-ResNet
Closest papers by meaning
- Spike-driven Transformer V2: Meta Spiking Neural Network Architecture Inspiring the Design of Next-generation Neuromorphic Chips (ICLR 2024 · similarity 0.88)
- Exploring the Limitations of Layer Synchronization in Spiking Neural Networks (NeurIPS 2026 · similarity 0.87)
- Enhanced Self-Distillation Framework for Efficient Spiking Neural Network Training (NeurIPS 2025 · similarity 0.87)
- Temporal Flexibility in Spiking Neural Networks: Towards Generalization Across Time Steps and Deployment Friendliness (ICLR 2025 · similarity 0.87)
- TS-SNN: Temporal Shift Module for Spiking Neural Networks (ICML 2025 · similarity 0.87)
- Are Conventional SNNs Really Efficient? A Perspective from Network Quantization (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.