Research map / Vision-language and multimodal / Multimodal LLMs

VisPlay: Self-Evolving Vision-Language Models

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Reinforcement learning for visual reasoning

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes VisPlay, a self-play GRPO framework where one VLM splits into an image-conditioned Questioner and a Reasoner that improves on unlabeled images using uncertainty/diversity rewards and majority-vote pseudo-labels.

Problem

Improving vision-language models' reasoning without human-annotated labels or task-specific verifiable rewards, which are costly and hard to scale.

Approach

A single base VLM alternates between a Questioner that is rewarded for generating questions the Reasoner is uncertain about (plus diversity penalty) and a Reasoner trained by GRPO against majority-vote pseudo-labels on curated moderate-confidence questions, co-evolving over iterations.

Key results

• Qwen2.5-VL-3B average accuracy rises from 30.61 (base) to 44.16 after iteration 1 and 47.27 at iteration 3.
• Qwen2.5-VL-7B average improves from 40.41 to 48.61 at iteration 3; MiMo-VL-7B from 43.56 to 45.69.
• Qwen2.5-VL-3B HallusionBench score rises from 32.81 to 94.95 by iteration 2.

Surprising findings

• Reasoner accuracy on iteration-1 questions improves while pseudo-label accuracy declines (72 to 61), reflecting increasing question difficulty.
• Standard GRPO with human-labeled data outperforms VisPlay on the 7B model average (50.7 vs 48.6), with VisPlay trailing on several tasks.

Limitations

• Experiments limited to models below 10B parameters (Qwen2.5-VL and MiMo-VL families).
• No definitive verification of self-generated pseudo-label data quality; errors may accumulate.

Experimental setup

Datasets: Vision-47K

Benchmarks: MM-Vet MMMU RealWorldQA VisNumBench MathVerse MATH-Vision HallusionBench

Metrics: accuracy average accuracy

Baselines: Standard GRPO with human-labeled Vision-47K VisPlay (challenger) Base model

Builds on: GRPO RLVR R-Zero Vision-Zero

Closest papers by meaning

• VICO: Visual Environments Co-Evolving for Vision-Language Model Reasoning (NeurIPS 2026 · similarity 0.90)
• Agent0-VL: Exploring Self-Evolving Agent for Tool-Integrated Vision-Language Reasoning (ICML 2026 · similarity 0.90)
• Vision-Zero: Scalable VLM Self-Improvement via Strategic Gamified Self-Play (ICLR 2026 · similarity 0.90)
• More Thought, Less Accuracy? On the Dual Nature of Reasoning in Vision-Language Models (ICLR 2026 · similarity 0.89)
• TTRV: Test-Time Reinforcement Learning for Vision Language Models (CVPR 2026 · similarity 0.89)
• DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning (ICLR 2026 · similarity 0.89)