Research map / Graph learning / Graph self-supervised learning
Graph World Model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes the Graph World Model (GWM), a unified world model that represents multi-modal states as graphs with action nodes, implemented as a token-based (GWM-T) and embedding-based (GWM-E) variant, and evaluates one model across six task families.
Problem
World models handle unstructured data but cannot exploit ubiquitous structured/graph data, and existing graph foundation models are limited to predefined graph learning tasks, not multi-modal or interdisciplinary tasks.
Approach
Model the world state as a multi-modal graph and each action as an action node that queries target nodes (intended, via graph structure; unintended, via embedding similarity); aggregate neighbor information either by token-level text message passing (GWM-T) or embedding-level multi-hop GCN-style aggregation with a trained MLP projector (GWM-E), then decode with an LLM or Stable Diffusion.
Key results
- A single GWM-E achieves 33.32 overall accuracy on LongBench v2 with 2k context, beating GPT-4o mini (128k) at 29.01.
- GWM-E reaches 96.23 accuracy on Multi-Modal-Paper matching, versus 51.77 for Contrastive MLP.
- GWM-E obtains 92.13 F1 on ALFWorld planning, above T5 FT at 91.82; GWM-E outperforms GWM-T on 5 of 7 tasks with roughly 5-10x fewer tokens.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GWM-E instead of GWM-T | GWM · LongBench v2 | Accuracy: 29.40 -> 33.32 | Helped |
| Use GWM-E instead of GWM-T | GWM · Baby recommendation | Recall: 70.84 -> 76.72 | Helped |
| Use GWM-E instead of GWM-T | GWM · Sports recommendation | Recall: 84.29 -> 88.78 | Helped |
| Use GWM-E instead of GWM-T | GWM · AgentClinic | F1 Score: 48.20 -> 35.56 | Hurt |
| Use GWM-E instead of GWM-T | GWM · ALFWorld | F1 Score: 87.42 -> 92.13 | Helped |
| Add graph information / multi-hop aggregation (ablation) | GWM-E · Six tasks | Average performance: +20% minimum relative gain on graph-related tasks | Helped |
| Increase the number of graph hops excessively (ablation) | GWM-E · Six tasks | Average performance | Hurt |
| Train on other tasks, then test on RAG without task-specific training | GWM · LongBench v2 | Accuracy | Helped |
| Fine-tune with 10% of target-task data | GWM · Agent and RAG | Performance | Helped |
Surprising findings
- Zero-shot GWM on RAG outperforms the Single Data-trained model, suggesting cross-task transfer helps tasks with limited data.
- On Goodreads image generation, the domain-specific INSTRUCTG2I baseline beats GWM variants on CLIP and DINOv2 scores.
Limitations
- Currently supports only text, table, and image modalities.
- Implementation focuses on homophilous graphs; non-homophilous structures are not handled.
- Context length limited to 2k tokens for GWM-T, and over-smoothing arises with more hops.
Experimental setup
Datasets: Goodreads Multi-Modal-Paper Amazon Baby Amazon Sports Amazon Clothing Cora PubMed HIV AgentClinic-NEJM LongBench v2 ALFWorld
Benchmarks: Goodreads Multi-Modal-Paper Amazon Baby Amazon Sports Amazon Clothing Cora PubMed HIV AgentClinic-NEJM LongBench v2 ALFWorld
Metrics: CLIP score DINOv2 Accuracy Recall F1 Score BERT-Score
Baselines: SD-1.5 SD-1.5 FT ControlNet INSTRUCTG2I Contrastive MLP CLIP CLIP FT FREEDOM LightGCN MMGCN GRCN GCN
Builds on: Genie LLAGA OFA GraphRAG latent diffusion models LLaVA
Closest papers by meaning
- GLBench: A Comprehensive Benchmark for Graph with Large Language Models (NeurIPS 2024 · similarity 0.86)
- GOFA: A Generative One-For-All Model for Joint Graph Language Modeling (ICLR 2025 · similarity 0.86)
- Talk like a Graph: Encoding Graphs for Large Language Models (ICLR 2024 · similarity 0.86)
- GraphVLM: Benchmarking Vision Language Models for Multimodal Graph Learning (CVPR 2026 · similarity 0.86)
- Can Large Language Models Analyze Graphs like Professionals? A Benchmark, Datasets and Models (NeurIPS 2024 · similarity 0.86)
- $\texttt{G1}$: Teaching LLMs to Reason on Graphs with Reinforcement Learning (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.