Research map / Probabilistic and scientific modeling / Dynamical systems
Time-Evolving Dynamical System for Learning Latent Representations of Mouse Visual Neural Activity
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TE-ViDS, a sequential latent variable model that splits latents into contrastively-trained external (stimulus) and KL-regularized internal (state) parts to decode natural scenes/movies from mouse visual cortex spikes.
Problem
Latent variable models for neural activity rarely model temporal dynamics explicitly, and few target visual cortex under naturalistic stimuli, leaving stimulus-relevant latent representations of visual neural activity poorly extracted.
Approach
A sequential VAE with GRU state factors that splits latents into deterministic "external" variables shaped by a time-offset NT-Xent contrastive loss and stochastic "internal" variables regularized by a time-dependent learned prior, with swap-based reconstruction.
Key results
- On natural scene decoding, TE-ViDS reaches 50.86% on Mouse 1 vs 32.81% for the best baseline (Swap-VAE) and achieves the top score on 4 of 5 mice.
- On natural movie frame decoding (1s window), TE-ViDS reaches 65.38% on Mouse 2 vs 52.76% for CEBRA.
- Ablations show removing both contrastive loss and swap operation drops Mouse 1 natural scene decoding from 50.86% to 6.88%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TE-ViDS instead of the strongest listed alternative | mouse visual neural decoding · Natural scenes across five mice | decoding score (%) | Helped |
| Use TE-ViDS instead of the strongest listed alternative | mouse visual neural decoding · Natural movie frames across five mice | decoding score (%) | Mixed |
| Add negative samples to the contrastive loss (ablation) | TE-ViDS · Natural scenes, Mouse 1 | decoding score (%): 21.85\(\pm\)1.86 -> 50.86\(\pm\)0.81 | Helped |
| Add contrastive loss (ablation) | TE-ViDS · Natural scenes, Mouse 1 | decoding score (%): 30.17\(\pm\)1.74 -> 50.86\(\pm\)0.81 | Helped |
| Add the swap operation (ablation) | TE-ViDS · Natural scenes, Mouse 1 | decoding score (%): 28.80\(\pm\)0.86 -> 50.86\(\pm\)0.81 | Helped |
| Add both contrastive loss and swap operation (ablation) | TE-ViDS · Natural scenes, Mouse 1 | decoding score (%): 6.88\(\pm\)0.64 -> 50.86\(\pm\)0.81 | Helped |
| Use a time-dependent prior rather than a temporal-independent prior (ablation) | TE-ViDS · Natural scenes, Mouse 1 | decoding score (%): 28.97\(\pm\)1.16 -> 50.86\(\pm\)0.81 | Helped |
| Use a GRU rather than a Vanilla RNN (ablation) | TE-ViDS · Natural scenes, Mouse 1 | decoding score (%): 47.64\(\pm\)0.78 -> 50.86\(\pm\)0.81 | Helped |
| Use a GRU rather than an LSTM (ablation) | TE-ViDS · Natural scenes, Mouse 1 | decoding score (%): 49.97\(\pm\)0.50 -> 50.86\(\pm\)0.81 | No clear effect |
| Keep the recurrent module (ablation) | TE-ViDS · Natural scenes, Mouse 1 | decoding score (%): 33.32\(\pm\)0.99 -> 50.86\(\pm\)0.81 | Helped |
| Include both external and internal latent representations (ablation) | TE-ViDS · Natural scenes, Mouse 1 | decoding score (%): 45.41\(\pm\)1.16 -> 50.86\(\pm\)0.81 | Helped |
| Add negative samples to the contrastive loss (ablation) | TE-ViDS · Natural movie frames, Mouse 1 | decoding score (%): 11.27\(\pm\)0.36 -> 13.88\(\pm\)0.19 | Helped |
Surprising findings
- CEBRA, which encodes temporal relationships, performs poorly on natural scene decoding (1-5%), suggesting fixed temporal filters are less suitable for static scenes.
- Internal-only latents give near-chance decoding (about 1.5-2.5% for scenes), while external-only nearly matches full model on some mice.
Limitations
- Internal latent variables cannot be quantitatively interpreted without recorded internal states or behavior.
- Substantial variability in decoding performance across individual mice.
- Evaluated only on one public mouse visual dataset with passive viewing.
Experimental setup
Datasets: Allen Brain Observatory Visual Coding (Neuropixels subset, 5 mice) synthetic non-temporal dataset (RealNVP-generated) synthetic Lorenz temporal dataset
Benchmarks: Allen Brain Observatory natural scenes decoding Allen Brain Observatory natural movie frame decoding synthetic non-temporal and Lorenz datasets (R^2 reconstruction)
Metrics: KNN decoding accuracy (%) R^2 reconstruction score
Baselines: PCA LFADS pi-VAE Swap-VAE CEBRA
Builds on: Swap-VAE LFADS CEBRA pi-VAE
Closest papers by meaning
- Modeling Dynamic Neural Activity by combining Naturalistic Video Stimuli and Stimulus-independent Latent Factors (NeurIPS 2025 · similarity 0.90)
- Flexible Context-Driven Sensory Processing in Dynamical Vision Models (NeurIPS 2024 · similarity 0.88)
- Multi-Modal Latent Variables for Cross-Individual Primary Visual Cortex Modeling and Analysis (AAAI 2025 · similarity 0.88)
- Probabilistic Decomposed Linear Dynamical Systems for Robust Discovery of Latent Neural Dynamics (NeurIPS 2024 · similarity 0.87)
- Dynamical modeling of nonlinear latent factors in multiscale neural activity with real-time inference (NeurIPS 2025 · similarity 0.87)
- Long-Range Feedback Spiking Network Captures Dynamic and Static Representations of the Visual Cortex under Movie Stimuli (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.