AI Paper Atlas

Research map / Vision-language and multimodal / Multimodal LLMs

Learning Where to Look: Observation Policy Optimization for Thinking with Images

Junfeng Wang, Jiawei Liu, Yongchao Xu, Tao Jiang, jiangbo Ai, Jin Zhang, Chong Wang, Qixing Zhang

NeurIPS 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes observation policy optimization to train vision-language models to choose better zoom/crop observation actions when thinking with images, motivated by error analysis showing many failures stem from incorrect observations.

Problem

Complex visual question answering with high-resolution images, where vision-language models must actively acquire local visual evidence by zooming into regions; many failures come from incorrect observation actions (failing to localize key regions or obtaining insufficient crops).

Approach

Treats the choice of where to look (observation actions such as crops) as a learnable policy and optimizes it directly, rather than optimizing only the final answer; details beyond the abstract are not stated.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
train with OPO instead of base Qwen3-VL-2B with zoom toolQwen3-VL-2B · V*overall accuracy: 75.9 -> 85.9Improved
train with OPO instead of base Qwen3-VL-8BQwen3-VL-8B · HR-Bench 8Koverall accuracy: 76.5 -> 80.8Improved
OPO instead of DeepEyes-style outcome GRPOQwen3-VL-2B · HR-Bench 8Koverall accuracy: 71.1 -> 76.4Improved
OPO instead of DeepEyes-style GRPOQwen2.5-VL-7B · HR-Bench 8Koverall accuracy: 68.0 -> 68.1No clear effect
OPO instead of DeepEyes-style GRPO on V* spatialQwen3-VL-8B · V* Spatialaccuracy: 93.4 -> 90.8Hurt
add UOB and ELAA (vs plain outcome-level GRPO) (ablation)Qwen3-VL-2B · HR-Bench 4Koverall accuracy: 71.5 -> 78.4Improved
add ELAA on top of UOB branching (ablation)Qwen3-VL-2B · HR-Bench 8Koverall accuracy: 73.8 -> 76.4Improved
branch on UOB without ELAA vs no branching (ablation)Qwen3-VL-2B · V*overall accuracy: 82.7 -> 83.8Improved
use uncertainty-guided event selection instead of random branching (ablation)Qwen3-VL-2B · HR-Bench 8Koverall accuracy: 73.0 -> 76.4Improved
add crop redundancy filtering (ablation)Qwen3-VL-2B · V*overall accuracy: 84.8 -> 85.9Improved
rollout budget M=4,B=1 instead of M=8,B=2 (ablation)Qwen3-VL-2B · HR-Bench 8Koverall accuracy: 76.4 -> 72.5Hurt
base trajectories B=4 instead of B=2 (M=8) (ablation)Qwen3-VL-2B · HR-Bench 4Koverall accuracy: 78.4 -> 77.1Hurt

Earlier methods this paper compares with

Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.

Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy