Research map / Vision-language and multimodal / Multimodal LLMs

Generative Visual Code Mobile World Models

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Multimodal LLM benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes gWorld, open-weight VLMs (8B, 32B) that predict the next mobile GUI state as renderable web code, trained on a 260K-sample code-based dataset built from offline trajectories, and introduces the MWMBench benchmark.

Problem

Mobile GUI world models must predict next screen states with both visual fidelity and precise text, but pixel-based image generators produce illegible text and copy inputs, while text-based WMs lose visual detail and existing visual pipelines (VIMO) are multi-stage, closed, and slow.

Approach

Reformulate visual GUI world modeling as next-state generation in renderable web code by a single VLM, fine-tuned on data created by relabeling next-state images to code via a frontier model and adding reasoning traces produced with look-ahead to the ground-truth next state.

Key results

• gWorld-32B achieves 79.6% average IAcc. across six MWMBench benchmarks, and gWorld-8B 74.9%, versus 52.5% for Qwen3-VL-32B and 67.4% for GLM-4.6V 106B.
• Using gWorld-8B as the world model in M3A with a Gemini 2.5 Flash backbone raises average step-wise accuracy to 69.9% from 62.3% for M3A alone and 47.5% for Qwen3-VL-8B as the world model.
• Average end-to-end latency is 0.55s for gWorld-8B and 1.30s for gWorld-32B versus 160s reported for VIMO.

Surprising findings

• Image-generation models achieve high visual similarity but low IAcc. (10.9 to 29.1%), often learning near-identity copying of the input state.
• Qwen3-VL 235B-A22B underperforms the much smaller Llama 4 402B and GLM-4.6V in IAcc. despite its size, reflecting that base VLMs without fine-tuning generate render failures at high rates.

Limitations

• Data generation relies on a closed frontier model (Gemini 3 Flash) for code relabeling and reasoning synthesis.
• IAcc. is a VLM-as-a-judge metric, and the human study covers only 100 samples.
• Policy experiments use an oracle-style M3A setup with ground-truth action included among K=3 candidates, which isolates selection rather than full policy performance.

Experimental setup

Datasets: Android in the Wild (AitW) GUIOdyssey AndroidControl Android Multi-annotation Expo (AMEX) AndroidWorld KApps

Benchmarks: MWMBench-AitW MWMBench-GUIOdyssey MWMBench-AndroidControl MWMBench-AMEX MWMBench-AndroidWorld MWMBench-KApps

Metrics: Instruction Accuracy (IAcc.) Similarity (DINO embedding cosine) Render Fail rate step-wise accuracy

Baselines: Qwen-Image-Edit 20B Emu3.5 34B Llama 4 109B-A17B Llama 4 402B-A17B Qwen3 VL 8B Qwen3 VL 32B Qwen3 VL 235B-A22B GLM-4.6V 106B VIMO

Builds on: VIMO Qwen3-VL M3A DINO

Closest papers by meaning

• Code2World: A GUI World Model via Renderable Code Generation (NeurIPS 2026 · similarity 0.90)
• ShowUI: One Vision-Language-Action Model for GUI Visual Agent (CVPR 2025 · similarity 0.88)
• Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents (ICLR 2025 · similarity 0.88)
• OS-ATLAS: Foundation Action Model for Generalist GUI Agents (ICLR 2025 · similarity 0.87)
• WorldCoder-Bench: Benchmarking Physically Grounded 3D World Synthesis (NeurIPS 2026 · similarity 0.87)
• JanusCoder: Towards a Foundational Visual-Programmatic Interface for Code Intelligence (ICLR 2026 · similarity 0.86)