Research map / Multi-agent systems and games / LLM agents
GameWorlds: Towards Standardized and Verifiable Evaluation of Multimodal Game Agents
Mingyu Ouyang, Siyuan Hu, Qinghong Lin, Hwee Tou Ng, Mike Zheng Shou
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase context memory rounds from 0 to 2 (ablation) | Qwen3-VL-235B-A22B Generalist · GameWorld | overall PG (%): 30.0 -> 30.6 | Improved |
| increase context memory rounds from 0 to 2 (ablation) | Qwen3-VL-235B-A22B CUA · GameWorld | overall PG (%): 30.3 -> 28.7 | Hurt |
| increase memory rounds from 0 to 2 (compute cost) (ablation) | Qwen3-VL-235B-A22B Generalist · GameWorld | sec/step: 5.5 -> 8.6 | Hurt |
| use semantic Generalist interface instead of raw CUA control | Qwen3-VL-235B-A22B / Qwen3-VL-30B-A3B · GameWorld repeated runs | overall PG (%): 30.4 vs 30.1 (235B); 30.9 vs 30.7 (30B) | No clear effect |
| run game in real time (no pause during inference) instead of paused (ablation) | Qwen3-VL-235B-A22B · GameWorld-RT | overall PG (%): 30.1 -> 34.0 (Generalist) | No clear effect |
| scale model from 30B-A3B to 235B-A22B (ablation) | Qwen3-VL Generalist · GameWorld-RT | overall PG (%): 32.9 -> 34.0 | Improved |
| scale model from 30B-A3B to 235B-A22B | Qwen3-VL CUA · GameWorld | overall PG (%): 30.8 -> 31.4 | No clear effect |
| compare best agents with human novice player under same 100-action budget | Gemini-3-Flash-Preview Generalist · GameWorld | overall PG (%): 64.1 (novice) vs 41.9 | Hurt |
Closest papers by meaning
- Steve-Eye: Equipping LLM-based Embodied Agents with Visual Perception in Open Worlds (ICLR 2024 · similarity 0.90)
- GameDevBench: Evaluating Agentic Capabilities Through Game Development (ICML 2026 · similarity 0.89)
- From Multimodal LLMs to Generalist Embodied Agents: Methods and Lessons (CVPR 2025 · similarity 0.89)
- VS-Bench: Evaluating VLMs for Strategic Abilities in Multi-Agent Environments (CVPR 2026 · similarity 0.89)
- VirtualEnv: A Platform for Embodied AI Research (AAAI 2026 · similarity 0.89)
- OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
