Research map / Probabilistic and scientific modeling / Dynamical systems
Dynamical Modeling of Behaviorally Relevant Spatiotemporal Patterns in Neural Imaging Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SBIND, a two-phase ConvRNN-plus-self-attention dynamical model that learns behaviorally relevant and residual dynamics directly from raw widefield calcium and functional ultrasound images.
Problem
Modeling high-dimensional neural imaging data (widefield calcium, functional ultrasound) and disentangling behaviorally relevant spatiotemporal dynamics, while existing neural-behavioral models rely on preprocessing (PCA, LocaNMF) that can discard behaviorally relevant information.
Approach
Combines convolutional recurrent networks (local spatial-temporal dependencies) with patch-based multi-head self-attention inside the recurrence (global dependencies), operating directly on raw image frames, and uses a two-phase learning scheme: a first ConvRNN is trained to decode behavior, and a second ConvRNN models residual neural image dynamics.
Key results
- x
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add behavior-supervised first-phase latent learning (ablation) | SBIND · WFCI 1 | behavior MSE: 0.5413 ± 0.0185 -> 0.4955 ± 0.0254 | Helped |
| Add behavior-supervised first-phase latent learning (ablation) | SBIND · WFCI 2 | behavior F1-score: 0.3985 ± 0.0194 -> 0.4569 ± 0.0036 | Helped |
| Add self-attention to the recurrence mapping (ablation) | SBIND · WFCI 1 | neural MSE: 0.0552 ± 0.0013 -> 0.0414 ± 0.0029 | Helped |
| Add self-attention to the recurrence mapping (ablation) | SBIND · WFCI 2 | neural MSE: 0.1948 ± 0.0047 -> 0.1644 ± 0.0090 | Helped |
| Add recurrence mapping (ablation) | SBIND · WFCI 1 | behavior MSE: 0.7866 ± 0.0447 -> 0.4955 ± 0.0254 | Helped |
| Add recurrence mapping (ablation) | SBIND · WFCI 2 | neural MSE: 0.2150 ± 0.0032 -> 0.1644 ± 0.0090 | Helped |
| Replace convolutional mappings with MLPs (ablation) | SBIND · WFCI 1 | neural MSE: 0.0414 ± 0.0029 vs 0.1239 ± 0.0040 | Hurt |
| Increase self-attention patch size (ablation) | SBIND · WFCI neural prediction | neural prediction performance | Helped |
Surprising findings
- Preprocessing with LocaNMF or PCA did not uniformly help; DPAD's behavior decoding did not improve with preprocessing, and CEBRA's neural prediction was very poor relative to SBIND.
Limitations
- Model is trained on data from a single session or animal; multi-session learning not explored.
- Behavior is only used during training; evaluation is one-step-ahead prediction.
- Tuning of latent channels and patch size per dataset required.
Experimental setup
Datasets: WFCI 1 (public widefield calcium imaging) WFCI 2 (public widefield calcium imaging) fUSI non-human primate memory-guided saccade dataset (public)
Benchmarks: WFCI 1 WFCI 2 fUSI
Metrics: MSE R^2 F1-score accuracy AUC
Baselines: DPAD CEBRA STNDT TNDM MLP-SBIND LDA with PCA SBIND-Unsup
Builds on: DPAD CEBRA PSID LocaNMF
Closest papers by meaning
- FAST-Brain: A Flow-Aligned Spatio-Temporal Surrogate Brain Model (NeurIPS 2026 · similarity 0.86)
- Dynamical modeling of nonlinear latent factors in multiscale neural activity with real-time inference (NeurIPS 2025 · similarity 0.86)
- Modeling Latent Neural Dynamics with Gaussian Process Switching Linear Dynamical Systems (NeurIPS 2024 · similarity 0.86)
- A scalable generative model for dynamical system reconstruction from neuroimaging data (NeurIPS 2024 · similarity 0.86)
- Identifying Neural Dynamics Using Interventional State Space Models (ICML 2025 · similarity 0.85)
- BLEND: Behavior-guided Neural Population Dynamics Modeling via Privileged Knowledge Distillation (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.