Research map / Vision-language and multimodal / Multimodal LLMs

CogniVerse: Revolutionizing Multi-Modal Retrieval-Augmented Generation with Cognitive Reflection and Geometric Reasoning

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Benchmarks and analysis of MLLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes CogniVerse, an MMRAG framework combining a cognitive reflection gate for retrieval necessity, hyperbolic embedding alignment with spectral knowledge-graph refinement, and an optimal-transport-augmented generation loss, reporting gains over MMCoQA and other baselines on three MMQA benchmarks.

Problem

Existing multi-modal RAG systems retrieve noisy or irrelevant content, misalign cross-modal embeddings, retrieve statically regardless of query need, and produce answers that are locally accurate but globally incoherent.

Approach

Combines a learned retrieval-necessity gate and relevance filter with hyperbolic (Lorentz) embedding alignment, spectral Laplacian-based subgraph selection over knowledge graphs, and a generation loss mixing cross-entropy with a Wasserstein term; the paper frames these as theoretically grounded but the theorems are informal.

Key results

• On Encyclopedic-VQA, CogniVerse reports 84.3% accuracy vs. 78.5% for MMCoQA (the strongest baseline in the table), with coherence 0.91 and retrieval precision 78.4%.
• On MultiModalQA, CogniVerse reports 82.7% accuracy and 76.8% retrieval precision, versus 75.9% and 69.2% for MMCoQA.
• Ablation on MultiModalQA: removing the Cognitive Reflection Module drops accuracy from 82.7% to 76.4%; the paper's ablation table shows the Optimal Transport Loss removal leaves retrieval precision unchanged at 76.8%.

Surprising findings

• Ablation table shows removing the optimal transport loss leaves retrieval precision identical to the full model (76.8%), and the text claims a coherence drop that is inconsistent with the 'w/o' accuracy being 79.3%.
• Main-text claims of robustness (80.1% accuracy under noise) and the reported 'WebQA' accuracy (79.5%) appear in different tables with inconsistent setups, and the abstract's 'lower latency' claim rests on latency differences of only 0.01-0.08 s.

Limitations

• Theorems are informal: the convergence claim assumes convexity of the geodesic loss in hyperbolic space, and the generation theorem is a sketch without rigorous proof.
• Wasserstein 'global' loss is described only loosely (W2 between empirical answer distributions), with no clear implementation details.
• Baselines are reimplemented or described vaguely; no code, weights, or data release is stated.

Experimental setup

Datasets: Encyclopedic-VQA MultiModalQA WebQA Wikipedia Wikidata

Benchmarks: Encyclopedic-VQA MultiModalQA WebQA

Metrics: accuracy coherence score (RoBERTa embedding cosine) retrieval precision (human-annotated) latency (s)

Baselines: CLIP-ViT-L BLIP-2 MuRAG MMCoQA GraphRAG

Builds on: MuRAG MMCoQA GraphRAG LLaVA

Closest papers by meaning

• Beyond Global Similarity: Multi-Conditional Retrieval for Fine-Grained Cross-Modal Understanding (CVPR 2026 · similarity 0.88)
• Generative Multi-Modal Knowledge Retrieval with Large Language Models (AAAI 2024 · similarity 0.87)
• Simple is Effective: The Roles of Graphs and Large Language Models in Knowledge-Graph-Based Retrieval-Augmented Generation (ICLR 2025 · similarity 0.87)
• SLQ: Bridging Modalities via Shared Latent Queries for Retrieval with Frozen MLLMs (ICML 2026 · similarity 0.87)
• MC-Search: Evaluating and Enhancing Multimodal Agentic Search with Structured Long Reasoning Chains (ICLR 2026 · similarity 0.87)
• Harnessing Large Language Models for Knowledge Graph Question Answering via Adaptive Multi-Aspect Retrieval-Augmentation (AAAI 2025 · similarity 0.87)