Research map / Vision-language and multimodal / Human and embodied modeling
Flexible Context-Driven Sensory Processing in Dynamical Vision Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds DCnet, a small biologically motivated recurrent convolutional visual network with E/I populations and low-rank top-down modulation from its own cue representations, and shows it solves a cue-delay-count visual search task and replicates human psychophysics reaction-time effects.
Problem
How high-level, spatially non-specific cue representations flexibly modulate low-level sensory dynamics for context-driven visual search, and how to model the resulting reaction-time behavior in a stimulus-computable, biologically realistic network.
Approach
A trainable recurrent visual network with separate excitatory and inhibitory populations, learnable cell-type time constants, and multiplicative low-rank top-down modulation computed from the model's own earlier sensory responses to cues; output entropy serves as a model reaction-time proxy.
Key results
- On vis-count held-out scenes, DCnet reaches 99% (color), 95% (shape), and 73% (conjunction) accuracy, outperforming implicit DNN baselines.
- On novel cues and scenes, DCnet attains 55% accuracy while the best baselines drop to chance (16.67%).
- DCnet reproduces psychophysics trends: output entropy differs significantly by distractor heterogeneity and, for low target-distractor difference, rises linearly with distractor count (Pearson r = 0.55).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DCnet's learned low-rank top-down feedback (ablation) | DCnet · vis-count | accuracy: 38% | Helped |
| Remove inhibition from DCnet (ablation) | DCnet · noise-driven network activity | Lag-1 autocorrelation | Hurt |
| Add inhibition (ablation) | DCnet · noise-driven network activity | dynamic range | Helped |
| Lesion modulation in early sensory areas (ablation) | DCnet · vis-count color trials | task performance | Hurt |
| Lesion modulation in late sensory areas (ablation) | DCnet · vis-count shape trials | task performance | Hurt |
| Increase distractor heterogeneity | DCnet fine-tuned for feature search · distractor heterogeneity feature search | output entropy: Mann–Whitney U = 302947., p < .001; Mann–Whitney U = 301376., p < .001 | Mixed |
| Increase distractor count when target-distractor color difference is high | DCnet fine-tuned for color search · target-distractor color difference search | output entropy: Pearson’s r = −0.11 | No clear effect |
| Increase distractor count when target-distractor color difference is low | DCnet fine-tuned for color search · target-distractor color difference search | output entropy: Pearson’s r = 0.55 | Helped |
| Use the EDL objective instead of the primary fine-tuning objective (ablation) | DCnet · target-distractor color difference search | accuracy: 95% -> 86% | Hurt |
| Increase distractor count on low target-distractor color difference trials | DCnet fine-tuned with EDL · target-distractor color difference search | model RT: Pearson’s r = 0.92 | Helped |
| Increase distractor count on high target-distractor color difference trials | DCnet fine-tuned with EDL · target-distractor color difference search | model RT: Pearson’s r = 0.2 | No clear effect |
Surprising findings
- Training DCnet without the top-down feedback mechanism drops performance to 38%.
- Model entropy does not show the human finding of slower reaction times for target-absent trials relative to target-present trials.
Limitations
- No long-range inter-area feedback and no compartmental separation of feedforward and feedback inputs.
- Output-entropy reaction time metric is simple and does not match the human target-absent RT pattern.
- GPT-4 zero-shot evaluation was small (30 trials, 37% success).
Experimental setup
Datasets: vis-count (generated from CLEVR-style renders) CIFAR-10 synthetic L/T feature search stimuli synthetic color disc search stimuli
Benchmarks: vis-count CIFAR-10 classic feature search psychophysics (distractor heterogeneity, color difference)
Metrics: accuracy output entropy as reaction time proxy Mann-Whitney U Pearson correlation lag-1 autocorrelation dynamic range
Baselines: 6-layer CNN ResNet-18 ViT-B/16 Swin-T Convolutional RNN GPT-4
Builds on: GRU ResNet Clevr evidential deep learning
Closest papers by meaning
- NEvo: Neural-Guided Evolutionary Video Synthesis for Dynamic Visual Selectivity (NeurIPS 2026 · similarity 0.89)
- Self-Attention-Based Contextual Modulation Improves Neural System Identification (ICLR 2025 · similarity 0.88)
- Time-Evolving Dynamical System for Learning Latent Representations of Mouse Visual Neural Activity (NeurIPS 2025 · similarity 0.88)
- Long-Range Feedback Spiking Network Captures Dynamic and Static Representations of the Visual Cortex under Movie Stimuli (NeurIPS 2024 · similarity 0.88)
- Geometry of naturalistic object representations in recurrent neural network models of working memory (NeurIPS 2024 · similarity 0.88)
- Complex priors and flexible inference in recurrent circuits with dendritic nonlinearities (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.