Research map / Architectures and efficiency / Spiking networks
Pretraining with Re-parametrized Self-Attention: Unlocking Generalizationin SNN-Based Neural Decoding Across Time, Brains, and Tasks
Yuqi Yang, Tengjun Liu, Haiyan Zhang, Wang Ruixue, Xuchao Chen, Mingkang Li, Yansong Chua, Nenggan Zheng, Shaomin Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add CST re-parameterized attention block instead of spike-driven transformer block (ablation) | RAT SNN-SS · Monkey C05 2025 | R2 x 100: 65.21 -> 66.71 | Improved |
| add CST re-parameterized attention block instead of spike-driven transformer block (ablation) | RAT SNN-SS · Monkey C05 2022 | R2 x 100: 80.09 -> 81.58 | Improved |
| use CST attention block rather than 3-hidden-layer fully connected recurrent SNN (ablation) | RAT SNN-SS · Monkey C05 2022 | R2 x 100: 79.47 -> 81.58 | Improved |
| enlarge MLP output dimension from 192 to 512 via surrogate shortcut (ablation) | RAT SNN-SS · Monkey C05 2025 | R2 x 100: 66.61 -> 66.71 | Improved |
| add structural re-parameterization (ablation) | RAT SNN-SS · Monkey C05 | R2 x 100: 67.58 -> 81.58 | Improved |
| add structural re-parameterization (ablation) | RAT SNN-SS · Monkey C05 | training epochs to converge: 545 -> 179 | Improved |
| add cross-session pretraining (ablation) | RAT SNN · NHP Monkey I | R2 x 100: 72.22 -> 74.26 | Improved |
| add cross-condition pretraining over single-session (ablation) | RAT SNN · NHP Monkey L | R2 x 100: 66.30 -> 70.40 | Improved |
| add cross-session pretraining (ablation) | RAT SNN · NLB RTT (Monkey I) | R2 x 100: 76.34 -> 78.55 | Improved |
| prune RAT SNN-CC (ablation) | RAT SNN-CC · NHP average | R2 x 100: 72.23 -> 71.96 | No clear effect |
| prune RAT SNN-CC (ablation) | RAT SNN-CC · NHP effective ACs | effective ACs: 65307 -> 21020 | Improved |
| add cross-condition pretraining to unseen subject and task (ablation) | RAT SNN · RTT-joystick unseen | training epochs | Improved |
Closest papers by meaning
- A Scalable, Causal, and Energy Efficient Framework for Neural Decoding with Spiking Neural Networks (NeurIPS 2025 · similarity 0.91)
- Leveraging unlabelled data for generalizable neural population decoding (NeurIPS 2026 · similarity 0.89)
- Temporal Flexibility in Spiking Neural Networks: Towards Generalization Across Time Steps and Deployment Friendliness (ICLR 2025 · similarity 0.89)
- STAA-SNN: Spatial-Temporal Attention Aggregator for Spiking Neural Networks (CVPR 2025 · similarity 0.89)
- Sign Gradient Descent-based Neuronal Dynamics: ANN-to-SNN Conversion Beyond ReLU Network (ICML 2024 · similarity 0.88)
- Temporal Misalignment in ANN-SNN Conversion and its Mitigation via Probabilistic Spiking Neurons (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
