Research map / Video / Video segmentation and tracking
Aligning Neuronal Coding of Dynamic Visual Scenes with Foundation Vision Models
Rining Wu, Feixiang Zhou, Ziwei Yin, Jian Liu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use DINOv2 prior plus causal 3D temporal convolutions and spikes alignment (Vi-ST) instead of I3D+MSTCN | retinal neural encoding model · Mov1 -> Mov2 cross-video | mean CC: 0.108 -> 0.334 | Improved |
| use Vi-ST instead of CRNN on same-video prediction | retinal neural encoding model · Mov1 -> Mov1 | mean CC: 0.857 -> 0.789 | Hurt |
| use Vi-ST instead of DINOv2+MSTCN on same-video prediction | retinal neural encoding model · Mov2 -> Mov2 | mean CC: 0.672 -> 0.570 | Hurt |
| replace RMSE with Vi-ST loss (RMSE plus -ReLU penalty plus rolling SoftDTW) (ablation) | Vi-ST · cross-video Mov1 and Mov2 | mean CC and SD-KL | Improved |
| use early-layer DINOv2 features as prior instead of deeper layers (ablation) | Vi-ST · cross-video | mean CC | Improved |
| remove the ViT prior (ablation) | Vi-ST · cross-video | mean CC | Hurt |
| use very deep ViT features as prior (ablation) | Vi-ST · cross-video | mean CC | No clear effect |
| add Spikes Alignment module (ablation) | Vi-ST · cross-video | mean CC | Improved |
| fuse RGC receptive field via 3D AdaLN Zero instead of direct addition (ablation) | Vi-ST · cross-video | robustness | Improved |
| increase number of jointly predicted RGCs (ablation) | Vi-ST · cross-video | mean CC | Improved |
Closest papers by meaning
- Transformer brain encoders explain human high-level visual responses (NeurIPS 2025 · similarity 0.89)
- Long-Range Feedback Spiking Network Captures Dynamic and Static Representations of the Visual Cortex under Movie Stimuli (NeurIPS 2024 · similarity 0.88)
- Time-Evolving Dynamical System for Learning Latent Representations of Mouse Visual Neural Activity (NeurIPS 2025 · similarity 0.88)
- Flexible Context-Driven Sensory Processing in Dynamical Vision Models (NeurIPS 2024 · similarity 0.88)
- Explaining the Implicit Neural Canvas: Connecting Pixels to Neurons by Tracing their Contributions (CVPR 2024 · similarity 0.88)
- Recurrent neural network dynamical systems for biological vision (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
