Research map / Vision-language and multimodal / Multimodal LLMs
ViperGPT: Visual Inference via Python Execution for Reasoning
Dídac Surís, Sachit Menon, Carl Vondrick
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| generate Python programs composing pretrained vision modules instead of using an end-to-end zero-shot model | ViperGPT · RefCOCO | IoU: 58.6 -> 72.0 (best prior zero-shot ReCLIP) | Improved |
| add simple_query (BLIP-2) module for non-decomposable questions | ViperGPT · GQA test-dev | accuracy: 44.7 -> 48.1 (vs zero-shot BLIP-2) | Improved |
| add llm_query (GPT-3) module for external knowledge | ViperGPT · OK-VQA | accuracy: 50.6 -> 51.9 (vs Flamingo) | Improved |
| apply the image-only program approach to video via ordered frames | ViperGPT · NExT-QA full set | accuracy: 60.0 vs supervised HiTeA 63.1; hard split T 49.8 vs 48.6 | Mixed |
| make a vision module or Python function nonfunctional by intervention (default output) (ablation) | ViperGPT programs · RefCOCO | mIoU drop | Mixed |
Later papers that compared with this method
10 later accepted papers report ViperGPT as a baseline in their results tables; 10 of them report a better number on most of the results they share with it.
- pySpatial: Generating 3D Visual Programs for Zero-Shot Spatial Reasoning (ICLR 2026 · better on 2 of 2 shared results)
- VipAct: Visual-Perception Enhancement via Specialized VLM Agent Collaboration and Tool-use (AAAI 2026 · better on 2 of 2 shared results)
- NePTune: A Neuro-Pythonic Framework for Tunable Compositional Reasoning on Vision-Language (ICLR 2026 · better on 1 of 1 shared result)
- Visual Agentic AI for Spatial Reasoning with a Dynamic API (CVPR 2025 · better on 1 of 1 shared result)
- What Makes a Maze Look Like a Maze? (ICLR 2025 · better on 1 of 1 shared result)
- NAVER: A Neuro-Symbolic Compositional Automaton for Visual Grounding with Explicit Logic Reasoning (ICCV 2025 · better on 1 of 1 shared result)
- Recursive Visual Programming (ECCV 2024 · better on 5 of 5 shared results)
- MoReVQA: Exploring Modular Reasoning Models for Video Question Answering (CVPR 2024 · better on 4 of 4 shared results)
2 more not listed.
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- BLIP-2 — BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation (ICML 2022) · better on 2 of 2 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- CodeV: Code with Images for Faithful Visual Reasoning via Tool-Aware Policy Optimization (CVPR 2026 · similarity 0.89)
- Multi-step Visual Reasoning with Visual Tokens Scaling and Verification (NeurIPS 2025 · similarity 0.87)
- Learning to Reason Iteratively and Parallelly for Complex Visual Reasoning Scenarios (NeurIPS 2024 · similarity 0.87)
- CodeDance: A Dynamic Tool-integrated MLLM for Executable Visual Reasoning (CVPR 2026 · similarity 0.87)
- No Labels, No Problem: Training Visual Reasoners with Multimodal Verifiers (ICLR 2026 · similarity 0.87)
- ViUniT: Visual Unit Tests for More Robust Visual Programming (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
