Research map / Vision-language and multimodal / Multimodal LLMs

GraphVLM: Benchmarking Vision Language Models for Multimodal Graph Learning

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: VLM capability analysis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces GraphVLM, a benchmark evaluating VLMs for multimodal graph node classification across three roles (encoder, aligner, predictor) on six Amazon/Reddit datasets, finding that fine-tuned VLM predictors perform best.

Problem

VLMs' capacity to reason over multimodal graphs (nodes with text and image attributes linked by relations) is underexplored, and existing MMGL methods lack a unified evaluation across GNN, LLM, and VLM backbones.

Approach

Organizes VLM use in multimodal graph learning into three functional roles (VLM-as-Encoder, VLM-as-Aligner, VLM-as-Predictor) and benchmarks them under a unified protocol, including LoRA-fine-tuned VLM predictors with structure-aware prompt or latent-space neighbor fusion.

Key results

• Qwen-VL-7B with SFT and latent-space fusion of text+image neighbors reaches 59.82% on CDs and 86.67% on Reddit, the best VLM-predictor results.
• Fine-tuning Qwen-VL-7B on Movies raises accuracy from 12.56% (zero-shot) to 54.18%, a 4.3x improvement.
• LLaGA with latent-space alignment gives the top accuracy on Movies (50.61%), Grocery (86.83%) and CDs (56.29%), while MLaGA leads on Toys (80.00%) and Arts (89.79%).

Surprising findings

• Zero-shot VLMs perform very poorly on node classification (e.g., 12.56% on Movies for Qwen-VL-7B), far below fine-tuned versions.
• Structure-aware encoders (CLIP-F-S) did not consistently beat non-structure-aware CLIP/CLIP-F encoders, and structure cues had unstable effects on the VLM-as-Encoder role.

Limitations

• Evaluation limited to node classification on six datasets from two domains (e-commerce and Reddit).
• Structure-aware benefits are inconsistent across datasets; sparse graphs (Toys, Grocery) show unstable or negative gains.
• VLM-based methods incur much higher training and inference cost than GNN baselines.

Experimental setup

Datasets: Amazon Movies Amazon Toys Amazon Grocery Amazon Arts Amazon CDs Reddit

Benchmarks: Amazon Movies Amazon Toys Amazon Grocery Amazon Arts Amazon CDs Reddit

Metrics: node classification accuracy

Baselines: GCN GraphSAGE MLP MMGCN MGAT UniGraph2 GraphPrompter LLaGA GraphGPT GraphTranslator MLaGA

Builds on: CLIP MMGCN MGAT UniGraph2 LLaGA GraphGPT MLaGA

Closest papers by meaning

• Mario: Multimodal Graph Reasoning with Large Language Models (CVPR 2026 · similarity 0.91)
• Mosaic of Modalities: A Comprehensive Benchmark for Multimodal Graph Learning (CVPR 2025 · similarity 0.91)
• Scene Graph Thinking: Reinforcing Structured Visual Reasoning for Multimodal Large Language Models (ICML 2026 · similarity 0.90)
• Visual Graph Arena: Evaluating Visual Conceptualization of Vision and Multimodal Large Language Models (ICML 2025 · similarity 0.90)
• VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks (ICLR 2025 · similarity 0.89)
• GRAPHGPT-O: Synergistic Multimodal Comprehension and Generation on Graphs (CVPR 2025 · similarity 0.89)