Research map / Large language models / Language, safety and interpretability
LaVCa: LLM-assisted Visual Cortex Captioning
Takuya Matsuyama, Shinji Nishimoto, Yu Takagi
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use LaVCa instead of BrainSCUBA | captioning · NSD sentence-level, subj01 | prediction accuracy: 0.207 -> 0.246 | Improved |
| use LaVCa instead of BrainSCUBA | captioning · NSD image-level, subj01 | prediction accuracy: 0.188 -> 0.213 | Improved |
| add multiple keywords and Sentence Composer (ablation) | LaVCa · NSD sentence-level, subj01 | prediction accuracy: 0.205 -> 0.246 | Improved |
| add Sentence Composer (ablation) | LaVCa · subj01 hyperparameter ablation | prediction accuracy: 0.230 -> 0.246 | Improved |
| use an encoding model to select optimal images (ablation) | LaVCa · subj07 | prediction accuracy: 0.094 -> 0.218 | Improved |
| do not shuffle captions within ROI (ablation) | LaVCa · OFA | prediction accuracy: 0.0280 -> 0.0945 | Improved |
| increase optimal images from 5 to 50 (ablation) | LaVCa · subj01 | prediction accuracy: 0.239 -> 0.246 | Improved |
| increase optimal images from 50 to 100 (ablation) | LaVCa · subj01 | prediction accuracy: 0.246 -> 0.246 | No clear effect |
| increase keywords from 5 to 10 (ablation) | LaVCa · subj01 | prediction accuracy: 0.246 -> 0.241 | Hurt |
| swap CLIP-Vision for SigLIP2 (ablation) | LaVCa · subj01 | prediction accuracy: 0.246 -> 0.232 | Hurt |
| use gpt-4o instead of Llama3.1-70B for keyword extraction (ablation) | LaVCa · subj01 | prediction accuracy: 0.238 -> 0.246 | Improved |
| use ridge regression instead of gradient descent for encoding model (ablation) | encoding model · subj01 | correlation: 0.484 -> 0.500 | Improved |
Closest papers by meaning
- BrainSCUBA: Fine-Grained Natural Language Captions of Visual Cortex Selectivity (ICLR 2024 · similarity 0.89)
- Neuro-Vision to Language: Enhancing Brain Recording-based Visual Reconstruction and Language Interaction (NeurIPS 2024 · similarity 0.88)
- Correlating instruction-tuning (in multimodal models) with vision-language processing (in the brain) (ICLR 2025 · similarity 0.87)
- CLIP-MSM: A Multi-Semantic Mapping Brain Representation for Human High-Level Visual Cortex (AAAI 2025 · similarity 0.87)
- Meta-Learning an In-Context Transformer Model of Human Higher Visual Cortex (NeurIPS 2025 · similarity 0.86)
- LLaDA-MedV: Exploring Large Language Diffusion Models for Biomedical Image Understanding (CVPR 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
