Research map / Video / Video segmentation and tracking
NEvo: Neural-Guided Evolutionary Video Synthesis for Dynamic Visual Selectivity
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes NEvo, an evolutionary search over structured text prompts that drives a video diffusion generator, scored by a V-JEPA 2 dynamic brain encoding model, to synthesize videos that maximally activate target visual cortex ROIs.
Problem
Model-based synthesis of stimuli for probing visual cortex selectivity has been limited to static images, leaving dynamic visual selectivity (motion, social interaction) underexplored.
Approach
Combine a dynamic multi-layer V-JEPA 2 voxel-wise encoding model with genetic search over an interpretable Cartesian prompt space, using a two-stage image-then-image-to-video pipeline, instead of gradient-based synthesis that fails for video.
Key results
- NEvo stimuli reach on average 99.8% of the Moments in Time response and 95.8% of the localizer response in predicted ROI activation.
- Optimized videos outperform first-frame static controls, with mean improvement 0.27 ± 0.03 and 0.61 ± 0.05 in MT.
- Property-activation correlation for MT with biological motion r=0.84 and pSTS with joint action r=0.73.
Surprising findings
- Gradient-based BrainDiVE fails in the dynamic setting, with responses averaging 24.0% of natural MiT video responses.
- Optimized videos improve predicted activation even in ventral FFA (0.14 ± 0.02) relative to first-frame controls.
Limitations
- Synthesized stimuli may reflect encoding-model bias rather than true neural selectivity; needs human neuroimaging validation.
- Vision-only model cannot capture speech/audiovisual cues relevant to regions like aSTS.
- Structured prompt space may omit relevant visual dimensions.
Experimental setup
Datasets: BOLDMoments social interaction fMRI dataset Moments in Time
Benchmarks: FFA PPA EBA MT V3A pSTS Moments in Time
Metrics: predicted ROI activation (a.u.) Pearson correlation between annotated properties and predict ICC(C,1) Cohen's d bootstrap 95% CI
Baselines: BrainDiVE CLIP last-layer encoding model random search hill climbing dynamic functional localizers Moments in Time videos single-stage video search
Builds on: V-JEPA 2 BrainDiVE SDXL-Turbo LTX-Video
Closest papers by meaning
- Flexible Context-Driven Sensory Processing in Dynamical Vision Models (NeurIPS 2024 · similarity 0.89)
- Retrospective for the Dynamic Sensorium Competition for predicting large-scale mouse primary visual cortex activity from videos (NeurIPS 2024 · similarity 0.87)
- In Silico Mapping of Visual Categorical Selectivity Across the Whole Brain (NeurIPS 2025 · similarity 0.87)
- Reanimating Images using Neural Representations of Dynamic Stimuli (CVPR 2025 · similarity 0.87)
- Modeling Dynamic Neural Activity by combining Naturalistic Video Stimuli and Stimulus-independent Latent Factors (NeurIPS 2025 · similarity 0.87)
- Time-Evolving Dynamical System for Learning Latent Representations of Mouse Visual Neural Activity (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.