Research map / Vision-language and multimodal / Multimodal LLMs
ODE: Open-Set Evaluation of Hallucinations in Multimodal Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ODE, an open-set dynamic protocol that generates synthetic images from graph-sampled concept/attribute combinations to evaluate object existence and attribute hallucinations in MLLMs, arguing static benchmarks suffer from data contamination.
Problem
Static hallucination benchmarks for multimodal LLMs use fixed test data, which risks data contamination and inflated scores, so models may appear more reliable than they are.
Approach
A weighted concept graph (objects, state/action/quantity attributes, co-occurrence edges) is used to sample concept pairs under four distribution criteria (Standard, Long-tail, Random, Fictional); text-to-image models synthesize novel test images, which are filtered with an open-vocabulary detector and paired with existence/attribute yes-no and generative queries.
Key results
- Models such as MiniGPT-4 and LLaVA-1.5 score F1 around 80-90 on static benchmarks but decline on ODE Standard pairs; e.g., MiniGPT-4 existence F1 drops to 64.3 on ODE Standard.
- On ODE-Flux vs ODE-SD existence tasks, CogVLM accuracy differs by +51.4 points, suggesting generator-dependent difficulty.
- Fine-tuning LLaVA-1.5 on erroneous ODE samples raised AMBER existence-level F1 from 83.0 to 97.9 and reduced generative Hal from 36.1 to 28.5.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Evaluate on ODE-generated samples instead of static COCO2014 images | MLLM hallucination evaluation · COCO2014, Internet images, and ODE-generated images | model performance | Mixed |
| Fine-tune LLaVA-1.5 on ODE-identified erroneous samples | LLaVA-1.5 · AMBER generative task | CHAIR: 8.1 -> 6.5 | Helped |
| Fine-tune LLaVA-1.5 on ODE-identified erroneous samples | LLaVA-1.5 · AMBER generative task | Cover: 51.4 -> 50.4 | Hurt |
| Fine-tune LLaVA-1.5 on ODE-identified erroneous samples | LLaVA-1.5 · AMBER generative task | Hal: 36.1 -> 28.5 | Helped |
| Fine-tune LLaVA-1.5 on ODE-identified erroneous samples | LLaVA-1.5 · AMBER generative task | Cog: 4.1 -> 2.9 | Helped |
| Fine-tune LLaVA-1.5 on ODE-identified erroneous samples | LLaVA-1.5 · AMBER existence discrimination | F1: 83.0 -> 97.9 | Helped |
| Fine-tune LLaVA-1.5 on ODE-identified erroneous samples | LLaVA-1.5 · AMBER attribute discrimination | F1: 64.8 -> 68.0 | Helped |
| Train on Standard-distribution ODE samples rather than Long-tail samples | Fine-tuned LLaVA · ODE fine-tuning across distribution types | performance improvement | Helped |
| Train on Long-tail ODE samples rather than Random samples | Fine-tuned LLaVA · ODE fine-tuning across distribution types | performance improvement | Hurt |
| Use ODE-generated images instead of Internet images | MLLM hallucination evaluation · Internet and ODE-generated images | hallucination evaluation outcome | No clear effect |
| Evaluate on Random or Fictional concept combinations instead of Standard combinations | MLLMs · ODE object hallucination evaluation | hallucination rate | Hurt |
Surprising findings
- Synthetic ODE-SD images yielded higher accuracy than ODE-Flux images for several models, showing that generator choice strongly changes measured hallucination rates.
- COCO2014 images gave higher performance than internet and ODE images, which the authors read as contamination.
Limitations
- Evaluation covers only object existence and attribute hallucinations in yes/no VQA and short description tasks, with a small test set (808 images, 40 concept combinations).
- Synthetic images from text-to-image models may differ semantically from natural images, and quality filtering relies on an open-vocabulary detector with a fixed confidence threshold.
- Contamination claims are inferred from performance gaps rather than verified against training data.
Experimental setup
Datasets: AMBER COCO2014 POPE
Benchmarks: ODE (Standard, Long-tail, Random, Fictional distributions) AMBER
Metrics: CHAIR Cover Hal Cog Accuracy Precision Recall F1
Baselines: CHAIR POPE AMBER COCO2014 subset
Builds on: AMBER POPE DyVal Stable Diffusion
Closest papers by meaning
- Mitigating Multimodal Hallucinations via Gradient-based Self-Reflection (CVPR 2026 · similarity 0.89)
- THRONE: An Object-based Hallucination Benchmark for the Free-form Generations of Large Vision-Language Models (CVPR 2024 · similarity 0.88)
- LLM-Check: Investigating Detection of Hallucinations in Large Language Models (NeurIPS 2024 · similarity 0.88)
- PerturboLLaVA: Reducing Multimodal Hallucinations with Perturbative Visual Training (ICLR 2025 · similarity 0.88)
- The Curse of Multi-Modalities: Evaluating Hallucinations of Large Multimodal Models across Language, Visual, and Audio (NeurIPS 2025 · similarity 0.88)
- Detecting and Mitigating Hallucination in Large Vision Language Models via Fine-Grained AI Feedback (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.