Research map / Vision-language and multimodal / Multimodal LLMs

MMLongCite: A Benchmark for Evaluating Faithfulness of Long-Context Vision-Language Models

NeurIPS 2026 · Vision-language and multimodal › Multimodal LLMs · approach: VLM capability analysis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces MMLongCite, a 2,280-example benchmark (plus MMLongCite-HR high-resolution grid variant) that evaluates multimodal citation faithfulness of 28 long-context vision-language models from 16K to 128K tokens, using GPT-5.2 as judge.

Problem

Existing multimodal faithfulness and long-context benchmarks are limited in context length, document diversity, and task complexity, and mostly measure answer correctness rather than whether answers are grounded via citations in long multimodal contexts.

Approach

Repurposes eight existing datasets into image-only, interleaved image-text, and video-only long contexts with unique citation indices, scores citation precision/recall/F1 plus correctness with an LLM judge, and adds grid-stitched high-resolution variants to test fine-grained visual grounding.

Key results

• Qwen3.5-122B-A10B achieves the best open-source citation F1 on MMLongCite (e.g., 80.51 multi-source QA), while Gemini-3-Pro leads overall with near-flat trends across lengths.
• On MMLongCite-HR Hard setting, Gemini-3-Pro keeps 76.77 citation F1 on 2WikiMultiHopQA versus 20.39 for Qwen2.5-VL-72B-Ins.
• Small models show a correctness-citation decoupling; e.g., Qwen2.5-VL-7B-Ins gets 51.23 correctness but only 17.92 citation F1 on visual grounding.

Surprising findings

• Enabling reasoning (think) mode boosts correctness and citation precision but reduces citation recall for several models such as MiMo-VL-7B-RL.
• Medium-sized Qwen3.5-4B surpasses the 72B Qwen2.5-VL-72B in citation F1, suggesting training recipe matters beyond scale.

Limitations

• Limited sample size per context-length interval may constrain distributional representativeness.
• Relies on GPT-5.2 as LLM judge; reliability assessed only by a human evaluation on a subset of models and datasets.
• Claude and GPT models not evaluated due to API limits on images per request.

Experimental setup

Datasets: LongDocURL MMLongBench-Doc SlideVQA 2WikiMultiHopQA Visual Haystack MM-NIAH Video-MME LongVideoBench

Benchmarks: MMLongCite MMLongCite-HR

Metrics: citation precision citation recall citation F1 correctness

Baselines: Gemini-3-Pro Gemini-3-Flash Seed-2.0-Pro Seed-1.6 Qwen2.5-VL-72B-Ins Qwen3-VL-235B-A22B-Ins GLM-4.6V Qwen3.5-122B-A10B

Builds on: ALCE LongCite MCiteBench MM-NIAH Visual Haystack

Closest papers by meaning

• MMLongBench: Benchmarking Long-Context Vision-Language Models Effectively and Thoroughly (NeurIPS 2025 · similarity 0.94)
• Are We on the Right Way for Evaluating Large Vision-Language Models? (NeurIPS 2024 · similarity 0.90)
• MMIU: Multimodal Multi-image Understanding for Evaluating Large Vision-Language Models (ICLR 2025 · similarity 0.90)
• MMInference: Accelerating Pre-filling for Long-Context Visual Language Models via Modality-Aware Permutation Sparse Attention (ICML 2025 · similarity 0.89)
• Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs (CVPR 2024 · similarity 0.89)
• GroundingME: Exposing the Visual Grounding Gap in MLLMs through Multi-Dimensional Evaluation (CVPR 2026 · similarity 0.89)