Research map / Vision-language and multimodal / Multimodal LLMs

Synthesizing Multimodal Verifiable Game Data to Boost VLMs' General Reasoning

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Reinforcement learning for visual reasoning

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Game-RL and Code2Logic to synthesize verifiable video-game VQA data (GameQA, 30 games, ~140K questions) and shows GRPO training solely on game tasks improves VLMs' general vision-language reasoning.

Problem

Vision-language RL has focused on narrow domains like geometry and charts, leaving broader training scenarios underexplored; existing game benchmarks for VLMs are evaluation-only and not in a trainable VQA format.

Approach

Use LLM-generated game code as the basis for a data engine that auto-generates verifiable VQA tasks (templates plus solver reusing game transition logic), then train VLMs with GRPO on these game tasks with an LLM judge providing binary final-answer rewards.

Key results

• Game-RL-Qwen2.5-VL-7B improves general-benchmark average from 49.94 to 52.27 (+2.33) after GRPO on 5k GameQA samples.
• GRPO on GameQA gives out-of-domain game gains of +1.16 to +3.82 points average across models on the GameQA test set.
• SFT on GameQA lowers general benchmark averages (e.g., Qwen2.5-VL-7B 49.94 to 47.26), while GRPO avoids this degradation.

Surprising findings

• SFT on GameQA (and on Geo-Multi) degraded general benchmark performance, whereas GRPO on the same data improved it.
• SFT-then-RL underperformed pure RL on out-of-domain general benchmarks (-2.5% vs +2.33%).

Limitations

• SFT on reasoning processes did not yield satisfactory out-of-domain generalization.
• GameQA is single-turn static VQA; multi-turn interaction is not covered.
• Gains on general benchmarks are modest (about +1 to +3 points).

Experimental setup

Datasets: GameQA MAVIS Multimodal-Open-R1-8k MultiMath300K Geo170K

Benchmarks: GameQA test set MathVista MathVerse MMBench MMMU CharXiv MathVision MMMU-Pro

Metrics: accuracy

Baselines: MAVIS (GRPO) Multimodal-Open-R1 (GRPO) MultiMath (GRPO) GPT-4o Claude-3.5-Sonnet Gemini-2.5-Pro Qwen2.5-VL-72B InternVL3-78B SFT on GameQA Geo-Multi (SFT)

Builds on: GRPO DeepSeek-R1 MAVIS MultiMath

Closest papers by meaning

• Play to Generalize: Learning to Reason Through Game Play (ICLR 2026 · similarity 0.92)
• Vision-Zero: Scalable VLM Self-Improvement via Strategic Gamified Self-Play (ICLR 2026 · similarity 0.91)
• Code as Reward: Empowering Reinforcement Learning with VLMs (ICML 2024 · similarity 0.90)
• Perception-R1: Advancing Multimodal Reasoning Capabilities of MLLMs via Visual Perception Reward (ICLR 2026 · similarity 0.90)
• GFlowVLM: Enhancing Multi-step Reasoning in Vision-Language Models with Generative Flow Networks (CVPR 2025 · similarity 0.90)
• GameVerse: Can Vision-Language Models Learn from Video-based Reflection? (ICML 2026 · similarity 0.90)