Research map / Probabilistic and scientific modeling / Dynamical systems
Inferring stochastic low-rank recurrent neural networks from neural data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Fits stochastic low-rank RNNs to noisy neural data via variational sequential Monte Carlo, with exact fixed-point enumeration via hyperplane arrangements and evaluation on teacher-student, EEG, hippocampal and monkey reaching datasets.
Problem
Fitting low-rank recurrent neural networks, which have tractable dynamics, to noisy neural data so that the models are both interpretable and fit the observed data well.
Approach
Combines low-rank piecewise-linear RNNs with a stochastic latent state trained by variational sequential Monte Carlo (optimal or bootstrap proposal, with an optional CNN encoder), and proves that the number of fixed points is bounded via hyperplane arrangement theory, enabling exhaustive fixed-point search.
Key results
- On MC_Maze (NLB), the method with 36 latents achieves co-bps 0.3225 and vel R2 0.8479, beating GPFA and SLDS but slightly below LFADS (0.3554) and NDT (0.3597) on co-bps.
- The paper proves an upper bound on the number of linear regions of piecewise-linear low-rank RNNs (sum over r of D^r binom(N,r)) and gives an exact fixed-point search requiring O(N^R) linear solves for fixed R and D.
- Using the non-causal encoder, co-bps improves to 0.3407 and vel R2 to 0.8902 on MC_Maze.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use a non-causal CNN encoder instead of a causal encoder (ablation) | Ours on MC_Maze · MC_Maze | co-bps: 0.3225 -> 0.3407 | Helped |
| use a non-causal CNN encoder instead of a causal encoder (ablation) | Ours on MC_Maze · MC_Maze | vel R2: 0.8479 -> 0.8902 | Helped |
| use a non-causal CNN encoder instead of a causal encoder (ablation) | Ours on MC_Maze · MC_Maze | PSTH R2: 0.5927 -> 0.5963 | Helped |
| use a non-causal CNN encoder instead of a causal encoder (ablation) | Ours on MC_Maze · MC_Maze | fp-bps: 0.2184 -> 0.2417 | Helped |
| add conditioning inputs at dim 5 (ablation) | Ours on monkey reaching · Monkey reaching | vel R2: 0.7897 ± 0.0687 -> 0.8589 ± 0.0493 | Helped |
| add conditioning inputs at dim 6 (ablation) | Ours on monkey reaching · Monkey reaching | vel R2: 0.8944 ± 0.0039 -> 0.9018 ± 0.0114 | Helped |
| use Ours instead of GPFA | MC_Maze · MC_Maze | co-bps: 0.2463 -> 0.3225 | Helped |
| use Ours instead of SLDS | MC_Maze · MC_Maze | vel R2: 0.7944 -> 0.8479 | Helped |
| use Ours instead of LFADS | MC_Maze · MC_Maze | co-bps: 0.3554 -> 0.3225 | Hurt |
| use Ours instead of LFADS | MC_Maze · MC_Maze | vel R2: 0.8906 -> 0.8479 | Hurt |
| use Ours instead of NDT | MC_Maze · MC_Maze | co-bps: 0.3597 -> 0.3225 | Hurt |
| use Ours instead of NDT | MC_Maze · MC_Maze | vel R2: 0.8897 -> 0.8479 | Hurt |
Surprising findings
- An approximate fixed-point search method can fail to find all fixed points in a rank-2, 128-unit RNN with 17 fixed points, while the exact method is guaranteed to find all.
- Full-rank LFADS sampled from the autonomous prior does not produce latents representing meaningful variability, and the controller-based version relies overly on data-inferred inputs.
Limitations
- Generation-focused method performs below state-of-the-art deep models (LFADS, NDT) on co-smoothing co-bps.
- Exact fixed-point search can be too costly in some scenarios, requiring fallback to approximate method.
- Latent dimensionality for MC_Maze (36) is higher than other networks considered, attributed to large number of conditions.
Experimental setup
Datasets: EEG motor imagery PhysioNet (S001R01) CRCNS HC-2 CRCNS HC-11 Monkey reaching MC_Maze (NLB) Synthetic teacher-student RNN data
Benchmarks: Neural Latents Benchmark MC_Maze EEG power spectrum and KL statistics HPC-2 spike statistics HPC-11 spike statistics
Metrics: co-bps vel R2 PSTH R2 fp-bps D_stsp D_H (Hellinger distance) firing rate and ISI statistics number of fixed points found
Baselines: LFADS NDT GPFA SLDS Spike smoothing approximate fixed-point search (Durstewitz CNS-2023)
Builds on: LFADS variational sequential Monte Carlo shPLRNN AutoLFADS
Closest papers by meaning
- Mechanistic Interpretability of RNNs emulating Hidden Markov Models (NeurIPS 2025 · similarity 0.89)
- Learning Reveals Invisible Structure in Low-Rank RNNs (NeurIPS 2026 · similarity 0.89)
- Probabilistic Decomposed Linear Dynamical Systems for Robust Discovery of Latent Neural Dynamics (NeurIPS 2024 · similarity 0.89)
- Inference of Neural Dynamics Using Switching Recurrent Neural Networks (NeurIPS 2024 · similarity 0.88)
- Modeling Latent Neural Dynamics with Gaussian Process Switching Linear Dynamical Systems (NeurIPS 2024 · similarity 0.88)
- Partial observation can induce mechanistic mismatches in data-constrained models of neural dynamics (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.