Research map / Vision-language and multimodal / Multimodal LLMs

PyVision-RL: Forging Open Agentic Vision Models via RL

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Reinforcement learning for visual reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes PyVision-RL, an RL framework for open-weight multimodal agents that uses Python as a dynamic tool, with oversampling-filtering-ranking rollouts and an accumulative tool reward to prevent interaction collapse, yielding PyVision-Image and PyVision-Video.

Problem

RL for agentic multimodal models suffers from interaction collapse, where models reduce tool calls and multi-turn reasoning; video agents also incur high visual token costs from uniform frame sampling.

Approach

Combines an oversampling-filtering-ranking rollout strategy (filter broken and zero-variance groups, rank groups by reward std) with an accumulative tool reward (0.1 per tool call, only when the answer is correct) and removes std normalization in GRPO advantages; for video, the full video is loaded only into a Python runtime so the agent samples frames on demand.

Key results

• PyVision-Image reaches 88.7 avg@32 on V* (vs 78.5 for Qwen2.5-VL-7B) and outperforms DeepEyes-v2 by +4.4 on DynaMath, +3.1 on MathVerse and +9.6 on WeMath.
• PyVision-Video reaches 44.0% on VSI-Bench using ~5K visual tokens per sample, versus 38.0% for Qwen2.5-VL-7B at ~45K tokens.
• Ablation at step 600: max turn budget 4 outperforms budget 2 by 1.93% on the seven-benchmark average.

Surprising findings

• Without the accumulative tool reward, the average tool call count rapidly decreases and stabilizes at a low value during RL.
• Many tool calls in PyVision-Image are 'no operation' calls that just re-plot the original image to revisit the visual hint.

Limitations

• Video agent's RL training uses only spatial reasoning data from SpaceR (15K samples).
• Python as primitive tool may access host file system; deployment needs safeguards (impact statement).

Experimental setup

Datasets: MMK12 GMAI-Reasoning ChartQA InfoVQA MMPR LLaVA-CoT LongVILA SpaceR DeepEyes Mini-o3 V-Thinker WeMath

Benchmarks: V* HRBench-4K HRBench-8K DynaMath MathVerse MathVision WeMath TIR-Bench VSI-Bench

Metrics: accuracy avg@32

Baselines: Qwen2.5-VL-7B Pixel-Reasoner Mini-o3 DeepEyes Thyme CodeV CodeDance CodeVision DeepEyes-v2 VITAL Video-R1 SpaceR

Builds on: GRPO PyVision DeepEyes Mini-o3

Closest papers by meaning

• DeepEyesV2: Toward Agentic Multimodal Model (ICLR 2026 · similarity 0.90)
• PeRL: Permutation-Enhanced Reinforcement Learning for Interleaved Vision-Language Reasoning (NeurIPS 2025 · similarity 0.89)
• ReWatch-R1: Boosting Complex Video Reasoning in Large Vision-Language Models through Agentic Data Synthesis (ICLR 2026 · similarity 0.89)
• Agentic Jigsaw Interaction Learning for Enhancing Visual Perception and Reasoning in Vision-Language Models (ICLR 2026 · similarity 0.89)
• Perception-R1: Advancing Multimodal Reasoning Capabilities of MLLMs via Visual Perception Reward (ICLR 2026 · similarity 0.88)
• ReAgent-V: A Reward-Driven Multi-Agent Framework for Video Understanding (NeurIPS 2025 · similarity 0.88)