Research map / Vision-language and multimodal / Multimodal LLMs

MMRAG-RFT: Two-stage Reinforcement Fine-tuning for Explainable Multi-modal Retrieval-augmented Generation

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Unified any-modality language models · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MMRAG-RFT, a two-stage GRPO-based reinforcement fine-tuning framework on Qwen2.5-VL that first does point-wise relevance filtering and then jointly optimizes list-wise document ranking and answer generation with explainable chain-of-thought, evaluated on WebQA and MultimodalQA.

Problem

Existing multi-modal retrieval-augmented generation methods return retrieval results and answers without explaining the reasoning behind document selection and response generation, and chain-of-thought SFT data for this setting is costly to obtain.

Approach

Applies rule-based GRPO point-wise Yes/No relevance training as a coarse filter, then reasoning-based GRPO with format, document-ID match, and BARTScore-based answer rewards so an MLLM produces explicit think/id/answer outputs for joint list-wise ranking and answer generation, without CoT supervision.

Key results

• On WebQA test, the method reaches QA 58.3, QA-FL 70.8, QA-Acc 76.3 and Retr 89.1, surpassing AETGA (QA 54.1) on the QA score.
• Trained only on the 5,000-sample Mini-WebQA, the method reaches QA 53.8 on WebQA test.
• On MultimodalQA (ImageQ subset), EM 71.7 and F1 79.7 versus RAMQA's 67.0 and 67.0.

Surprising findings

• Replacing stage-1 RFT with SFT drops Retr from 76.9 to 51.5, showing the point-wise RFT stage is critical for retrieval.
• Stage-2 SFT (with RFT stage 1) dropped QA from 50.9 to 34.9, indicating RFT matters for answer quality.

Limitations

• Relies on Mini-WebQA which is a filtered subset; full-data results on MultimodalQA use only ImageQ subset.
• Point-wise ranking ignores inter-document relations, which motivates but is only partially addressed by stage 2.
• Hardware and compute budget not reported.

Experimental setup

Datasets: WebQA MultimodalQA Mini-WebQA

Benchmarks: WebQA MultimodalQA

Metrics: Retr QA-FL QA-Acc QA EM F1

Baselines: VLP VLP + VinVL MuRAG SKURG PERQA VLMT RAMQA AETGA Q-only AutoRouting

Builds on: GRPO DeepSeek-R1 RAMQA Qwen2.5-VL

Closest papers by meaning

• Improving Retrieval-Augmented Generation through Multi-Agent Reinforcement Learning (NeurIPS 2025 · similarity 0.88)
• MMhops-R1: Multimodal Multi-hop Reasoning (AAAI 2026 · similarity 0.87)
• Generative Multi-Modal Knowledge Retrieval with Large Language Models (AAAI 2024 · similarity 0.87)
• MC-Search: Evaluating and Enhancing Multimodal Agentic Search with Structured Long Reasoning Chains (ICLR 2026 · similarity 0.86)
• CogniVerse: Revolutionizing Multi-Modal Retrieval-Augmented Generation with Cognitive Reflection and Geometric Reasoning (CVPR 2026 · similarity 0.86)
• MM-HELIX: Boosting Multimodal Long-Chain Reflective Reasoning with Holistic Platform and Adaptive Hybrid Policy Optimization (ICLR 2026 · similarity 0.86)