Research map / Vision-language and multimodal / Multimodal LLMs
Game Ground Bench: Probing the Limits of LVLMs in Complex Semantic Grounding Across Game Universes
Zhangyang Qi, Jinsong Li, Hongjian Wu, Jiaqi Wang, Hengshuang Zhao
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Game Ground Bench, a benchmark for evaluating LVLMs on complex semantic grounding in video game interfaces across multiple game universes.
Problem
Existing grounding benchmarks are inadequate for testing LVLMs on language grounding in complex interactive environments like video games: real-world datasets such as RefCOCO have a domain gap, GUI benchmarks lack the complexity of modern game interfaces, and existing game benchmarks are limited.
Approach
A game-specific grounding benchmark spanning multiple game universes that targets complex semantic referring expressions in modern game screenshots, designed to probe the limits of LVLMs.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train with GRPO reinforcement fine-tuning instead of SFT (500 steps) (ablation) | Game-R1 7B · GGBench FPS genre | Acc@50: 39.4 -> 43.4 | Improved |
| train with GRPO instead of SFT (500 steps), averaged over three genres (ablation) | Game-R1 7B · GGBench (FPS, Card, Race avg) | Acc@50: 39.2 -> 42.1 | Improved |
| use soft continuous IoU reward instead of binary IoU>=0.5 (ablation) | Game-R1 7B · GGBench (FPS, Card, Race avg) | Acc@50: 38.4 -> 42.1 | Improved |
| train on multiple genres instead of FPS only (ablation) | Game-R1 7B · GGBench (FPS, Card, Race avg) | Acc@50: 39.8 -> 42.1 | Improved |
| train on FPS only (ablation) | Game-R1 7B · GGBench FPS genre | Acc@50: 43.4 -> 45.7 | Improved |
| increase RL training steps from 300 to 500 (ablation) | Game-R1 7B · GGBench FPS genre | Acc@50: 41.2 -> 43.4 | Improved |
| RL fine-tune Qwen2.5-VL-7B on 500 game images with GRPO | Qwen2.5-VL-7B · GGBench (average) | Acc@50: 33.4 -> 45.8 | Improved |
| game-only GRPO fine-tuning | Qwen2.5-VL-7B · MMBench / OCRBench / MMVet / RefCOCO val | accuracy / score: MMBench 82.2 -> 82.8; OCRBench 888 -> 884; RefCOCO val 90.0 -> 90.1 | No clear effect |
Experimental setup
Benchmarks: Game Ground Bench
Builds on: RefCOCO
Closest papers by meaning
- Synthesizing Multimodal Verifiable Game Data to Boost VLMs' General Reasoning (ICLR 2026 · similarity 0.88)
- GameVerse: Can Vision-Language Models Learn from Video-based Reflection? (ICML 2026 · similarity 0.88)
- Are Large Vision Language Models Good Game Players? (ICLR 2025 · similarity 0.88)
- BALROG: Benchmarking Agentic LLM and VLM Reasoning On Games (ICLR 2025 · similarity 0.88)
- Vision-Zero: Scalable VLM Self-Improvement via Strategic Gamified Self-Play (ICLR 2026 · similarity 0.87)
- VS-Bench: Evaluating VLMs for Strategic Abilities in Multi-Agent Environments (CVPR 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
