Research map / Vision-language and multimodal / Multimodal LLMs
RFI: Rectified Flow Intervention for Mitigating Object Hallucination in Large Vision-Language Models
Junyu Cheng, Zhibiao Liang, Yidong Chen, Shuangyin Li
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Rectified Flow Intervention (RFI), a method to mitigate object hallucination in LVLMs, evaluated from an abstract-only view.
Problem
Object hallucination in large vision-language models, where generated text is inconsistent with the input image; existing dynamic mitigation approaches have limitations (the abstract is truncated, so the specific limitations are not fully stated).
Approach
Uses rectified flow to intervene on LVLM hidden representations to steer generation toward image-consistent content; exact mechanism not stated in the available abstract text.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add RFI rectified-flow intervention | LLaVA-v1.5 · POPE COCO random | F1: 81.33 -> 90.40 | Improved |
| add RFI rectified-flow intervention | LLaVA-v1.5 · POPE A-OKVQA adversarial | F1: 75.15 -> 82.00 | Improved |
| add RFI rectified-flow intervention | LLaVA-v1.5 and Qwen-VL · POPE (average over subsets) | F1: 7.59% (LLaVA-v1.5) and 6.90% (Qwen-VL) | Improved |
| replace static ICT vector with dynamic RFI | ICT on LLaVA-v1.5 · POPE A-OKVQA random | F1: 89.41 -> 91.15 | Improved |
| replace static ICT vector with dynamic RFI | ICT on Qwen-VL · POPE COCO random | F1: 89.20 -> 88.89 | Hurt |
| replace ICT with RFI | ICT on Qwen-VL · POPE COCO adversarial | F1: 84.42 -> 84.49 | No clear effect |
| replace ICT with RFI | ICT on LLaVA-v1.5 and Qwen-VL · MME hallucination subset | score: 27.91 points (LLaVA-v1.5) and 47.48 points (Qwen-VL) | Improved |
| replace ICT with RFI | ICT · MME count | score | Hurt |
| move intervention layer away from layer 15 (ablation) | RFI · POPE COCO random | F1 | Hurt |
| tune intensity alpha to about 5.0 (LLaVA) and 2.0 (Qwen-VL) (ablation) | RFI · POPE COCO random | F1 | Improved |
| add RFI at inference (100 new tokens) | LLaVA-v1.5 · Latency | seconds: 2.79 -> 3.05 | Hurt |
| add RFI at inference (5 new tokens) | LLaVA-v1.5 · Latency | seconds: 0.28 -> 0.53 | Hurt |
Closest papers by meaning
- Reducing Hallucinations in Large Vision-Language Models via Latent Space Steering (ICLR 2025 · similarity 0.94)
- Prefill-Time Intervention for Mitigating Hallucination in Large Vision-Language Models (CVPR 2026 · similarity 0.93)
- Mitigating Object Hallucination in Large Vision-Language Models via Image-Grounded Guidance (ICML 2025 · similarity 0.93)
- Dynamic Multimodal Activation Steering for Hallucination Mitigation in Large Vision-Language Models (ICLR 2026 · similarity 0.93)
- Look Carefully: Adaptive Visual Reinforcements in Multimodal Large Language Models for Hallucination Mitigation (ICLR 2026 · similarity 0.92)
- Detecting and Mitigating Hallucination in Large Vision Language Models via Fine-Grained AI Feedback (AAAI 2025 · similarity 0.92)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
