Research map / Reinforcement learning / Offline RL

Decision SpikeFormer: Spike-Driven Transformer for Decision Making

CVPR 2025 · Reinforcement learning › Offline RL · approach: Decision Transformer sequence models · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Decision SpikeFormer (DSFormer), a spike-driven transformer for offline RL via decision-transformer-style sequence modeling, using temporal and positional spiking self-attention plus progressive threshold-dependent normalization, evaluated on D4RL.

Problem

Offline reinforcement learning with spiking neural networks is energy-efficient in principle, but existing spiking transformers target vision and their spatial attention and normalization are ill-suited to sequence modeling of RL trajectories.

Approach

Replaces spatial spiking self-attention with temporal-concatenated attention (TSSA) and a local positional-bias attention (PSSA), and introduces PTBN, which transitions from LayerNorm-style normalization to BatchNorm so the trained model can be merged into linear layers and keep spiking behavior at inference.

Key results

• On D4RL MuJoCo, PSSA averages 78.8 versus DT's 74.7, with energy about 88.8 µJ versus 410.5 µJ for DT.
• On D4RL Adroit, PSSA averages 48.6 and TSSA 47.9, versus 27.8 for DT.
• The abstract reports 78.4% energy savings relative to ANN counterparts.

Measured effects reported in the paper

Surprising findings

• SpikeGPT and SpikeBERT perform poorly on offline RL, with SpikeBERT's integer shortcut values disrupting spike propagation.

Limitations

• No deployment on neuromorphic hardware; energy results are estimates.
• SNN architectures from NLP baselines perform poorly, so comparisons to other SNN offline-RL models are limited.
• Ablations are run only on the MuJoCo-Hopper tasks.

Experimental setup

Datasets: D4RL MuJoCo D4RL Adroit D4RL AntMaze

Benchmarks: D4RL MuJoCo D4RL Adroit D4RL AntMaze

Metrics: normalized score energy consumption (µJ)

Baselines: BC CQL DT FCNet SpikeGPT SpikeBERT

Builds on: Decision Transformer Spikformer tdBN MetaFormer Membrane Shortcut

Closest papers by meaning

• SpikingResformer: Bridging ResNet and Vision Transformer in Spiking Neural Networks (CVPR 2024 · similarity 0.88)
• GRSN: Gated Recurrent Spiking Neurons for POMDPs and MARL (AAAI 2025 · similarity 0.88)
• Adaptive Surrogate Gradients for Sequential Reinforcement Learning in Spiking Neural Networks (NeurIPS 2025 · similarity 0.88)
• Decision ConvFormer: Local Filtering in MetaFormer is Sufficient for Decision Making (ICLR 2024 · similarity 0.87)
• Signal-Adaptive Trust Regions for Gradient-Free Optimization of Recurrent Spiking Neural Networks (NeurIPS 2026 · similarity 0.86)
• SpikedAttention: Training-Free and Fully Spike-Driven Transformer-to-SNN Conversion with Winner-Oriented Spike Shift for Softmax Operation (NeurIPS 2024 · similarity 0.86)