AI Paper Atlas

Research map / Vision-language and multimodal / Multimodal LLMs

Game Ground Bench: Probing the Limits of LVLMs in Complex Semantic Grounding Across Game Universes

Zhangyang Qi, Jinsong Li, Hongjian Wu, Jiaqi Wang, Hengshuang Zhao

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: VLM capability analysis · 1 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces Game Ground Bench, a benchmark for evaluating LVLMs on complex semantic grounding in video game interfaces across multiple game universes.

Problem

Existing grounding benchmarks are inadequate for testing LVLMs on language grounding in complex interactive environments like video games: real-world datasets such as RefCOCO have a domain gap, GUI benchmarks lack the complexity of modern game interfaces, and existing game benchmarks are limited.

Approach

A game-specific grounding benchmark spanning multiple game universes that targets complex semantic referring expressions in modern game screenshots, designed to probe the limits of LVLMs.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
train with GRPO reinforcement fine-tuning instead of SFT (500 steps) (ablation)Game-R1 7B · GGBench FPS genreAcc@50: 39.4 -> 43.4Improved
train with GRPO instead of SFT (500 steps), averaged over three genres (ablation)Game-R1 7B · GGBench (FPS, Card, Race avg)Acc@50: 39.2 -> 42.1Improved
use soft continuous IoU reward instead of binary IoU>=0.5 (ablation)Game-R1 7B · GGBench (FPS, Card, Race avg)Acc@50: 38.4 -> 42.1Improved
train on multiple genres instead of FPS only (ablation)Game-R1 7B · GGBench (FPS, Card, Race avg)Acc@50: 39.8 -> 42.1Improved
train on FPS only (ablation)Game-R1 7B · GGBench FPS genreAcc@50: 43.4 -> 45.7Improved
increase RL training steps from 300 to 500 (ablation)Game-R1 7B · GGBench FPS genreAcc@50: 41.2 -> 43.4Improved
RL fine-tune Qwen2.5-VL-7B on 500 game images with GRPOQwen2.5-VL-7B · GGBench (average)Acc@50: 33.4 -> 45.8Improved
game-only GRPO fine-tuningQwen2.5-VL-7B · MMBench / OCRBench / MMVet / RefCOCO valaccuracy / score: MMBench 82.2 -> 82.8; OCRBench 888 -> 884; RefCOCO val 90.0 -> 90.1No clear effect

Experimental setup

Benchmarks: Game Ground Bench

Builds on: RefCOCO

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy