Research map / Large language models / Retrieval and knowledge
MMGraph-Agent: Agentic Multimodal RAG via Cache-Inspired Multimodal Knowledge HyperGraphs
Haoran Luo, Ziyue Zhu, Shangyang Wu, Linhao Luo, Yikai Guo, Qika Lin, Fangzhi Xu, Jiapu Wang, Xiaobao Wu, Yifan Zhu, Anh Tuan Luu
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MMGraph-Agent, an agentic multimodal RAG system that organizes knowledge as cache-inspired multimodal knowledge hypergraphs to address construction cost, context limits, and unified offline/online knowledge training.
Problem
Multimodal RAG suffers from high knowledge-graph construction cost, limited context windows, and lack of unified training across offline and online knowledge sources.
Approach
Combines a cache-inspired multimodal knowledge hypergraph for knowledge organization with an agentic retrieval process, aiming to unify offline and online knowledge sourcing under one training scheme. Details beyond the abstract are not available.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use MMGraph-Agent-3B over second-best baseline per dataset | Qwen2.5-VL-3B agent · 23 multimodal QA datasets | token F1: +11.6 F1 on average | Improved |
| MMGraph-Agent vs MMSearch-R1 on OOD datasets | Qwen2.5-VL-3B agent · 8 held-out OOD datasets | token F1: +13.0 F1 | Improved |
| add multimodal hypergraph (ablation) | MMGraph-Agent · K+Web | token F1: +16.6 | Improved |
| add cache-inspired memory (ablation) | MMGraph-Agent · K+Web | token F1: +6.4 | Improved |
| add offline-trained GRPO policy (ablation) | MMGraph-Agent · K+Web | token F1: +6.3 | Improved |
| add retrieval (ablation) | MMGraph-Agent · K+Web | token F1: +3.6 | Improved |
| use multimodal hypergraph instead of pairwise-edge KG (ablation) | MMGraph-Agent · 23 datasets | token F1: wins on 23/23 datasets | Improved |
| add L1 cache with C=5 per-bank budget (ablation) | MMGraph-Agent · long-chain episodes | tokens and F1: 67% fewer tokens, +3.4 F1 | Improved |
| replace LLM extraction with spaCy NER and frozen CLIP for graph construction | MMGraph-Agent · construction cost | F1 and API cost: overall F1 30.0 at zero API cost | Improved |
Closest papers by meaning
- Navigating Massive Visual Context in Retrieval-Augmented Generation via Multimodal Memory Graph (ICML 2026 · similarity 0.90)
- MC-Search: Evaluating and Enhancing Multimodal Agentic Search with Structured Long Reasoning Chains (ICLR 2026 · similarity 0.89)
- Simple is Effective: The Roles of Graphs and Large Language Models in Knowledge-Graph-Based Retrieval-Augmented Generation (ICLR 2025 · similarity 0.88)
- HyperGraphRAG: Retrieval-Augmented Generation via Hypergraph-Structured Knowledge Representation (NeurIPS 2025 · similarity 0.88)
- Beyond RAG for Agent Memory: Retrieval by Decoupling and Aggregation (NeurIPS 2026 · similarity 0.88)
- A Systematic Exploration of Knowledge Graph Alignment with Large Language Models in Retrieval Augmented Generation (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
