Research map / Vision-language and multimodal / Multimodal LLMs

FINER: MLLMs Hallucinate under Fine-grained Negative Queries

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Unified any-modality language models · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces FINER fine-grained negative query benchmarks (FINER-CompreCap, FINER-DOCCI) showing MLLMs hallucinate on multi-object/attribute/relation and wh-questions, and proposes FINER-Tuning, a DPO recipe on fine-grained positive/negative queries that improves four MLLMs on FINER, hallucination suites and general benchmarks.

Problem

MLLMs falsely accept fine-grained incorrect queries involving multiple objects, attributes, and relations, a failure mode underrepresented by existing coarse hallucination benchmarks.

Approach

Build MCQ benchmarks from scene graphs with four LLM-generated negatives per entity, filtered by an MLLM discriminator and entropy thresholds, and use these fine-grained positive and negative queries as DPO preference data (Pixmo-caption-derived) to post-train frontier MLLMs.

Key results

• FINER-Tuning on InternVL3.5-14B improves Multi-rel paired accuracy on FINER-CompreCap from 47.0 to 71.2 (+24.2).
• FINER-Tuning improves InternVL3.5-14B on MMHal-Bench hallucination rate from 11.0 to 10.0 and general average from 68.0 to 69.4 across six benchmarks.
• On DASH, FINER-Tuning boosts InternVL3.5-8B by 6.2 points (68.3 to 74.5).

Surprising findings

• SFT on positive-and-negative queries reduces Multi-obj performance by 36.7% relative to base, while DPO improves it.
• Hallucination-aware methods such as RLHF-V and LLaVA-RLHF score near chance on FINER-CompreCap Multi-rel (1.6% and 1.1%).

Limitations

• Large-scale FINER-DOCCI is not fully human-curated; some noise remains in negatives and relations.
• Rule-based MCQ templates may reduce question naturalness.
• Multi-rel subsets contain at most three relations.

Experimental setup

Datasets: Pixmo-caption CompreCap DOCCI

Benchmarks: FINER-CompreCap FINER-DOCCI DASH POPE RePOPE HallusionBench AMBER CRPE_R MMHal-Bench HaloQuest MMStar TextVQA

Metrics: paired accuracy accuracy hallucination rate score

Baselines: LRV-V2 LLaVA-RLHF RLHF-V OPA-DPO RLAIF-V Gemini-2.5-Flash InternVL-3.5-38B

Builds on: DPO POPE RLAIF-V CompreCap DOCCI

Closest papers by meaning

• CHiP: Cross-modal Hierarchical Direct Preference Optimization for Multimodal LLMs (ICLR 2025 · similarity 0.89)
• PostAlign: Multimodal Grounding as a Corrective Lens for MLLMs (ICLR 2026 · similarity 0.88)
• Coarse-to-Fine Highlighting: Reducing Knowledge Hallucination in Large Language Models (ICML 2024 · similarity 0.88)
• SEED-Bench: Benchmarking Multimodal Large Language Models (CVPR 2024 · similarity 0.88)
• ANAH-v2: Scaling Analytical Hallucination Annotation of Large Language Models (NeurIPS 2024 · similarity 0.88)
• Hallucination at a Glance: Controlled Visual Edits and Fine-Grained Multimodal Learning (NeurIPS 2025 · similarity 0.88)