Research map / Vision-language and multimodal / Multimodal LLMs
Reflective Instruction Tuning: Mitigating Hallucinations in Large Vision-Language Models
Jinrui Zhang, Teng Wang, Haigang Zhang, Ping Lu, Feng Zheng
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add reflective rationale learning with REVERIE to LLaVA-1.0-7b-lora | LLaVA-1.0-7b-lora · POPE | score: 71.1 -> 83.8 | Improved |
| add REVERIE to LLaVA-1.0-7b-lora | LLaVA-1.0-7b-lora · MME / MMBench | score: MME 819.8 -> 1168.1; MMBench 27.2 -> 55.4 | Improved |
| add REVERIE to LLaVA-1.5-7b-lora (vs adding only ScienceQA data) | LLaVA-1.5-7b-lora · ScienceQA-IMG | accuracy: 76.3 -> 80.5 | Improved |
| add REVERIE to LLaVA-1.5-7b-lora | LLaVA-1.5-7b-lora · POPE / MMBench / MM-Vet | score: POPE 86.6 -> 86.4; MMBench 67.9 -> 67.3 | No clear effect |
| add only ScienceQA response data (no rationales) to LLaVA-1.5 training (ablation) | LLaVA-1.5-7b-lora · MME / GQA | score: MME 1471.0 -> 1439.2; GQA 62.8 -> 60.7 | Hurt |
| add positive and negative rationales together versus response only (ablation) | REVERIE-1.0-7b-lora · POPE Adversarial | F1: 78.49 -> 80.31 | Improved |
| add only positive rationales (ablation) | REVERIE-1.0-7b-lora · POPE Random | Accuracy: 86.87 -> 85.60 | Hurt |
| add both rationales versus response only (ablation) | REVERIE-1.0-7b-lora · MMBench | accuracy: 51.70 -> 55.44 | Improved |
| add negative rationales on top of positive rationales (ablation) | REVERIE-1.0-7b-lora · GQA | accuracy: 35.19 (pos) -> 36.53 (pos+neg); neg only 36.77 | Improved |
| include explicit guidance 'Including any necessary facts or knowledge' in the rationale prompt (ablation) | REVERIE-1.0-7b-lora · MME | score: 1128.6 -> 1168.1 | Improved |
| add type-specific or length-control prompt constraints (ablation) | rationale prompt · MMB, POPE, GQA, MME | score: MME 1168.1 -> 1111.9 (type-specific); 1141.0 (plus length control) | Hurt |
| apply consistency-based filtering of rationale data (ablation) | REVERIE-1.0-7b-lora · MME | score: 1132.3 -> 1168.1 | Improved |
Closest papers by meaning
- Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning (ICLR 2024 · similarity 0.92)
- VES-RFT: Rewarding Visual Evidence Sensitivity to Mitigate Hallucinations in Large Vision-Language Models (CVPR 2026 · similarity 0.91)
- Visual Description Grounding Reduces Hallucinations and Boosts Reasoning in LVLMs (ICLR 2025 · similarity 0.90)
- More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning Models (NeurIPS 2025 · similarity 0.90)
- From Seeing to Thinking: Decoupling Perception and Reasoning Improves Post-Training of Vision-Language Models (ICML 2026 · similarity 0.90)
- Understanding the Role of Hallucination in Reinforcement Post-Training of Multimodal Reasoning Models (CVPR 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
