Research map / Vision-language and multimodal / Multimodal LLMs

Mario: Multimodal Graph Reasoning with Large Language Models

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Benchmarks and analysis of MLLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Mario, a two-stage framework for LLM-based reasoning on multimodal graphs: a topology-aware graph-conditioned VLM aligns text and image per node, then modality-adaptive instruction tuning with a learned router picks the best modality template per node.

Problem

LLM-based reasoning over multimodal graphs (nodes with text and image attributes) suffers from weak cross-modal consistency when using frozen VLM embeddings and from heterogeneous per-node modality preference that a fixed prompt template ignores.

Approach

A dual-tower VLM whose [CLS] tokens attend over all graph nodes with shortest-path-distance position bias, trained with symmetric InfoNCE; plus a Modality-Adaptive Prompt Router that chooses among text-only, image-only, and multimodal graph-context prompt templates per node, trained with a posterior-weighted LLM loss and a KL term.

Key results

• Mario-8B reaches 63.43% node classification accuracy on CDs versus 56.45% for the best baseline in the single-focus setting.
• In zero-shot transfer Toys→Movies, Mario-8B attains 41.00% NC accuracy and 86.60% LP accuracy, about 1.64× the best baseline NC accuracy.
• Mario-8B achieves 93.90% LP accuracy on Movies in the single-focus setting, outperforming Graph4MM at 90.24%.

Surprising findings

• Frozen-LLM Mario is only slightly below its LoRA-tuned version, with LoRA adding about 1.7–3.0 NC points.

Limitations

• Training requires three forward-backward passes per sample, one per template, increasing per-epoch cost by roughly 1.5–2×.
• Mario underperforms some baselines on certain Mix-Training cells (e.g., LLaGA on Movies NC), and the Mix-Training numbers drop relative to single-focus.

Experimental setup

Datasets: Amazon-Movies Reddit-S Amazon-CDs&Vinyl Amazon-Arts&Crafts Amazon-Toys Goodreads

Benchmarks: Movies Reddit CDs Arts Toys Goodreads

Metrics: accuracy

Baselines: GCN GATv2 GraphSAGE LLaGA GraphPrompter GraphGPT MLaGA Graph4MM UniGraph2 Qwen2.5-VL LLaVA1.5-13B

Builds on: CLIP LLaGA GraphGPT LoRA InfoNCE

Closest papers by meaning

• GraphVLM: Benchmarking Vision Language Models for Multimodal Graph Learning (CVPR 2026 · similarity 0.91)
• GRAPHGPT-O: Synergistic Multimodal Comprehension and Generation on Graphs (CVPR 2025 · similarity 0.91)
• Scene Graph Thinking: Reinforcing Structured Visual Reasoning for Multimodal Large Language Models (ICML 2026 · similarity 0.91)
• DiGraphHal-Bench: Evaluating Multimodal Large Language Models on Complex Directed Graphs (CVPR 2026 · similarity 0.90)
• VisionGraph: Leveraging Large Multimodal Models for Graph Theory Problems in Visual Context (ICML 2024 · similarity 0.90)
• Mosaic of Modalities: A Comprehensive Benchmark for Multimodal Graph Learning (CVPR 2025 · similarity 0.90)