Research map / Architectures and efficiency / Spiking networks

Spike2Former: Efficient Spiking Transformer for High-performance Image Segmentation

AAAI 2025 · Architectures and efficiency › Spiking networks · approach: Spiking transformers · 13 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Spike2Former adapts Mask2Former into a spiking transformer for semantic segmentation, using spike-driven deformable encoder/mask-embedding redesigns and a normalized integer LIF neuron to reach SOTA SNN mIoU on ADE20K, VOC2012 and Cityscapes.

Problem

Spiking neural networks perform poorly on image segmentation; directly converting complex segmentation architectures like Mask2Former into spiking versions causes severe spike degradation, performance loss and non-convergence.

Approach

Identify spike-degrading modules in Mask2Former (deformable attention encoder and mask embedding), redesign them with spike-driven deformable attention on spiked attention weights, a membrane embedding shortcut, and propose NI-LIF, which normalizes integer activations during training while remaining spike-driven at inference.

Key results

• On ADE20k Spike2Former reaches 46.3% mIoU at 68.0 mJ, versus 33.6% for the prior SNN SpikeFPN (88.1 mJ).
• On Pascal VOC2012 Spike2Former reaches 75.4% mIoU at 63.0 mJ (T x D = 1x4 variant), versus 61.1% for SpikeFPN at 179.4 mJ.
• On Cityscapes Spike2Former reaches 75.2% mIoU versus 66.5% for SCGNet-L; ablation shows direct I-LIF gives 37.9% mIoU versus 46.3% for NI-LIF on ADE20k.

Measured effects reported in the paper

Surprising findings

• Directly converting Mask2Former to a spiking version fails to converge (5.2% mIoU reported in ablation).
• Using multi-scale features as queries gives slightly higher mIoU (46.7%) but doubles energy consumption (136.5 mJ).

Limitations

• Performance remains below ANN Mask2Former (46.3% vs 47.2% on ADE20k) and the gap on Cityscapes/VOC is not fully closed
• Energy numbers are estimates from a formula rather than measured on neuromorphic hardware
• Multi-scale query variant gives higher accuracy at doubled energy; design trades accuracy for efficiency

Experimental setup

Datasets: ADE20k Cityscapes Pascal VOC2012 ImageNet-1k

Benchmarks: ADE20k Cityscapes Pascal VOC2012

Metrics: mIoU parameters (M) estimated power (mJ)

Baselines: SpikeFPN SpikeFCN SpikeDeepLab Spike Calibration SCGNet-L Mask2Former MaskFormer DeepLabv3+ I-LIF

Builds on: Mask2Former I-LIF Meta-SpikeFormer SpikeFPN SEW-ResNet

Closest papers by meaning

• Spike-driven Transformer V2: Meta Spiking Neural Network Architecture Inspiring the Design of Next-generation Neuromorphic Chips (ICLR 2024 · similarity 0.89)
• SpikeVideoFormer: An Efficient Spike-Driven Video Transformer with Hamming Attention and $\mathcal{O}(T)$ Complexity (ICML 2025 · similarity 0.88)
• Inference-Scale Complexity in ANN-SNN Conversion for High-Performance and Low-Power Applications (CVPR 2025 · similarity 0.88)
• SpikingResformer: Bridging ResNet and Vision Transformer in Spiking Neural Networks (CVPR 2024 · similarity 0.88)
• STEP: A Unified Spiking Transformer Evaluation Platform for Fair and Reproducible Benchmarking (NeurIPS 2025 · similarity 0.88)
• SpikedAttention: Training-Free and Fully Spike-Driven Transformer-to-SNN Conversion with Winner-Oriented Spike Shift for Softmax Operation (NeurIPS 2024 · similarity 0.87)