Research map / Vision-language and multimodal / Multimodal LLMs

Vision-Zero: Scalable VLM Self-Improvement via Strategic Gamified Self-Play

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Reinforcement learning for visual reasoning

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Vision-Zero, a label-free multi-agent "Who Is the Spy" visual self-play game, and Iterative-SPO, which alternates self-play clue training with GRPO-based decision training to improve VLMs on reasoning, chart, and vision-centric benchmarks.

Problem

VLM reinforcement learning depends on costly human-curated datasets and verification, limiting scalable self-improvement; self-play has not been explored for VLMs.

Approach

A visual social-deduction game where civilians see an image and the spy sees a blank image (or a modified image); players give clues and vote for the spy, with zero-sum rewards for clue stage and verifiable vote rewards for decision stage, alternated adaptively by an accuracy/n-a hysteresis controller.

Key results

• VisionZero-Qwen-7B (CLEVR) reaches 44.3 average on six reasoning/math benchmarks vs 41.1 for Qwen2.5-VL-7B and 42.9 for the best RLVR baseline (MM-Eureka).
• Win rate of Qwen2.5-VL-7B against untrained reference rises from 50% to 71% over CLEVR training.
• VisionZero-Qwen-7B (CLEVR) reaches 58.8 MMMU using 2k unlabeled samples and 127 A100-hours, versus MM-Eureka's 55.8 with 15k samples and about 700 A100-hours.

Surprising findings

• Vision-Zero with no math-specific training improves math and reasoning benchmarks more than RLVR baselines trained on thousands of math samples.
• Pure self-play plateaus early, and alternating with RLVR gives gains over both pure self-play and pure RLVR.

Limitations

• Gains on some benchmarks are modest (about 2-3 points average) and single-run variance is not reported.
• Reliance on GPT-4o for chart data generation and the spy-image construction limits the fully label-free claim.
• Win-rate and efficiency evaluations are against untrained references and self-measured costs.

Experimental setup

Datasets: CLEVR (rendered 2k pairs) ChartQA training images (1k) ImgEdit training images (1k) MM-Eureka training set

Benchmarks: MathVista MathVision WeMath MathVerse LogicVista DynaMath ChartXiV_RQ FunctionQA PaperQA ReachQA RealWorldQA MMVP

Metrics: accuracy win rate average token length

Baselines: R1-OneVision-7B MM-Eureka-Qwen-7B VLAA-Thinker-7B OpenVLThinker-7B ViGaL-Snake+Rotation vanilla GRPO on MM-Eureka data pure self-play pure RLVR

Builds on: GRPO SPIRAL ViGaL VLM-R1 Absolute Zero

Closest papers by meaning

• Synthesizing Multimodal Verifiable Game Data to Boost VLMs' General Reasoning (ICLR 2026 · similarity 0.91)
• GameVerse: Can Vision-Language Models Learn from Video-based Reflection? (ICML 2026 · similarity 0.90)
• VisPlay: Self-Evolving Vision-Language Models (CVPR 2026 · similarity 0.90)
• VLMs-Guided Representation Distillation for Efficient Vision-Based Reinforcement Learning (CVPR 2025 · similarity 0.89)
• Code as Reward: Empowering Reinforcement Learning with VLMs (ICML 2024 · similarity 0.88)
• Reinforcing VLAs in Task-Agnostic World Models (NeurIPS 2026 · similarity 0.88)