Research map / Vision-language and multimodal / Multimodal LLMs

MC-Search: Evaluating and Enhancing Multimodal Agentic Search with Structured Long Reasoning Chains

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces MC-Search, a 3,333-example benchmark for agentic multimodal RAG with long, step-wise annotated reasoning chains across five topologies, plus process-level metrics and Search-Align SFT on verified chains.

Problem

Existing multimodal RAG benchmarks use short 1-2 hop retrieval chains with no stepwise annotations, so they cannot assess long, adaptive, cross-modal agentic search reasoning in MLLMs.

Approach

A benchmark with golden step-wise trajectories (sub-question, retrieval modality, evidence, intermediate answer) filtered by HAVE (hop-wise utility and navigational checks), process-level metrics (HPS, RD, LLM-as-judge), and process-supervised SFT (Search-Align) on verified chains.

Key results

• Gemini-2.5-Pro achieves the highest F1 on most topologies (e.g., 47.61 F1 on Image-Initiated Chain), while Qwen2.5-VL-7B lags.
• Search-Align raises Qwen2.5-VL-7B F1 by about +13.7 on average and HPS by +16.0, and InternVL3.5-8B by about +2.8 F1 and +12.0 HPS.
• Image-modality coverage is weak without image input: InternVL3.5-8B image coverage drops from 63.84% to 0.66% when no image is in the query.

Surprising findings

• Under single-step RAG, open-source models often answer overconfidently while proprietary Gemini models abstain, and Gemini-2.5-Flash/Pro score negative ΔF1 on some topologies.
• Moderate over-retrieval (1-2 extra steps) can improve accuracy, while under-retrieval is more harmful.

Limitations

• Benchmark limited to science-free general knowledge domains from Wikipedia; authors plan extension to science and math.
• Relies on Gemini models for data generation, filtering, and LLM-as-judge evaluation, which may introduce model-specific bias.
• Retrieval restricted to top-1 per step from a local knowledge base.

Experimental setup

Datasets: Wikipedia text and images (MC-Search knowledge base)

Benchmarks: MC-Search

Metrics: F1 ΔF1 Golden F1 LLM-as-a-Judge Hit per Step Rollout Deviation

Baselines: GPT-4o-Mini Gemini-2.5-Flash Gemini-2.5-Pro Claude-3.7-Sonnet InternVL3.5-8B Qwen2.5-VL-7B single-step RAG two-hop traditional MM-RAG

Builds on: agentic RAG Search-Align

Closest papers by meaning

• MDK12-Bench: A Multi-Discipline Benchmark for Evaluating Reasoning in Multimodal Large Language Models (AAAI 2026 · similarity 0.90)
• AgentHop: A Diagnostic Benchmark for Agentic Multi-Hop Scientific Question Answering (NeurIPS 2026 · similarity 0.90)
• Retrieval is Not Enough: Enhancing RAG through Test-Time Critique and Optimization (NeurIPS 2025 · similarity 0.89)
• Iterative Self-Incentivization Empowers Large Language Models as Agentic Searchers (NeurIPS 2025 · similarity 0.89)
• Strategic Navigation or Stochastic Search? How Agents and Humans Reason Over Document Collections (ICML 2026 · similarity 0.89)
• Chain of Agents: Large Language Models Collaborating on Long-Context Tasks (NeurIPS 2024 · similarity 0.89)