Research map / Architectures and efficiency / RNNs and normalization
Generalizable, real-time neural decoding with hybrid state-space models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes POSSM, a hybrid model pairing POYO-style spike tokenization and cross-attention with a recurrent SSM/GRU backbone for fast, causal, generalizable neural decoding, pretrained on multi-dataset NHP data and transferred to human handwriting and speech decoding.
Problem
Neural decoders must be accurate, causal and low-latency for real-time BCI use, and also generalize to new sessions, subjects and tasks; RNNs are fast but do not generalize, while Transformers generalize but are computationally heavy.
Approach
Combine per-spike tokenization with a cross-attention encoder applied to 50 ms chunks, feed the resulting latents into a recurrent SSM/GRU backbone that carries hidden state across chunks, and use unit-identification and full finetuning to adapt multi-dataset pretrained models to new neural units.
Key results
- On NHP reaching, o-POSSM-S4D (FT) reaches R^2 0.9068 on new-animal CO and 0.8941 on the new-dataset session, matching or beating POYO-1 (FT).
- On human handwriting, o-POSSM-Mamba (FT) reaches 97.73% accuracy versus 95.82% for POYO-1 (FT) and 81.36% for PCA-KNN, showing cross-species transfer from monkey pretraining.
- On human speech, POSSM-GRU reaches 27.32% PER versus 30.06% for the GRU baseline, and up to 9x faster GPU inference than Transformers is reported.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Pretrain POSSM on multiple NHP datasets, then full-finetune on new NHP sessions | POSSM · NHP reaching, T – CO (6) | R²: 0.8453 ± 0.0200 -> 0.9068 ± 0.0170 | Helped |
| Use full finetuning rather than unit identification (ablation) | o-POSSM-S4D · NHP reaching, T – CO (6) | R²: 0.8763 ± 0.0259 -> 0.9068 ± 0.0170 | Helped |
| Pretrain on NHP data and full-finetune for human handwriting | POSSM-Mamba · Human handwriting | accuracy: 93.55 ± 4.53 -> 97.73 ± 2.13 | Helped |
| Use POSSM-GRU instead of the GRU baseline | Human speech decoder · Human speech | phoneme error rate (PER): 30.06 -> 27.32 | Helped |
| Remove Gaussian noise augmentation (ablation) | POSSM-GRU · Human speech | phoneme error rate (PER): 27.32 -> 29.70 | Hurt |
| Remove Gaussian noise augmentation (ablation) | GRU · Human speech | phoneme error rate (PER): 30.06 -> 39.16 | Hurt |
| Use bidirectional rather than unidirectional backbones (ablation) | POSSM-GRU · Human speech, no augmentation | phoneme error rate (PER): 29.70 -> 26.62 | Helped |
| Use 20 ms chunks instead of 50 ms chunks (ablation) | POSSM-GRU-SS · NHP reaching | R² | No clear effect |
| Omit delimiter tokens ([START] and [END]) (ablation) | POSSM · Neural decoding experiments | decoding performance | No clear effect |
| Pretrain on NHP data and full-finetune for human handwriting | POSSM-S4D · Human handwriting | accuracy: 92.11 ± 6.66 -> 97.25 ± 2.88 | Helped |
| Pretrain on NHP data and full-finetune for human handwriting | POSSM-GRU · Human handwriting | accuracy: 95.82 ± 3.41 -> 97.37 ± 2.32 | Helped |
Surprising findings
- Pretraining only on monkey motor-cortex data improved decoding of human imagined handwriting, indicating cross-species transfer of neural dynamics.
- POSSM-GRU without Gaussian noise augmentation still beats the GRU baseline with augmentation on speech decoding (29.70% vs 30.06% PER with augmentation for baseline).
Limitations
- Evaluations are offline; real-time online deployment is not tested.
- Speech decoding is performed without language model priors and uses binned spike counts rather than spike times.
Experimental setup
Datasets: Multi-lab NHP reaching datasets (CO, RT, Maze) NLB Maze NLB Area2 Human handwriting imagined characters dataset Human attempted speech dataset (public) Allen Visual Behaviour Neuropixels
Benchmarks: NHP reaching (CO, RT, Maze) R^2 human imagined handwriting accuracy human speech PER
Metrics: R^2 accuracy phoneme error rate inference time per chunk parameter count
Baselines: MLP S4D Mamba GRU POYO-1 NDT-2 PCA-KNN
Builds on: POYO PerceiverIO S4D Mamba NDT-2
Closest papers by meaning
- Leveraging unlabelled data for generalizable neural population decoding (NeurIPS 2026 · similarity 0.88)
- SPINT: Spatial Permutation-Invariant Neural Transformer for Consistent Intracortical Motor Decoding (NeurIPS 2025 · similarity 0.88)
- A Scalable, Causal, and Energy Efficient Framework for Neural Decoding with Spiking Neural Networks (NeurIPS 2025 · similarity 0.88)
- Time-Masked Transformers with Lightweight Test-Time Adaptation for Neural Speech Decoding (NeurIPS 2025 · similarity 0.87)
- Dynamical modeling of nonlinear latent factors in multiscale neural activity with real-time inference (NeurIPS 2025 · similarity 0.87)
- Few-shot Algorithms for Consistent Neural Decoding (FALCON) Benchmark (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.