Research map / Vision-language and multimodal / Multimodal LLMs

Grounding-IQA: Grounding Multimodal Language Model for Image Quality Assessment

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Benchmarks and analysis of MLLMs

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces Grounding-IQA, an IQA paradigm combining referring and grounding with quality assessment, plus the GIQA-160K dataset (automatically annotated) and the GIQA-Bench benchmark, and fine-tunes MLLMs on it.

Problem

Existing MLLM-based image quality assessment relies on general contextual descriptions and cannot give precise locations or answer fine-grained, region-level quality questions.

Approach

Integrates multimodal referring (position input) and grounding (position output) into IQA via two subtasks (grounding-IQA description and grounded VQA), with an automated annotation pipeline that uses Llama3, Grounding DINO, and Q-Instruct as an IQA filter to build box-grounded quality data from human-annotated descriptions; coordinates are discretized into grid-index tokens written as plain text.

Key results

• Grounding-IQA fine-tuned mPLUG-Owl2-7B reaches 0.7417 total VQA accuracy and 63.00 LLM-Score on GIQA-Bench, the best among compared models.
• Fine-tuning LLaVA-v1.5-7B on GIQA-160K raises GIQA-DES Tag-Recall from N/A (no grounding) to 0.5283 and total VQA accuracy from 0.4733 to 0.6850.
• Ablation shows box refinement (IQA-filter and box merge) improves GIQA-DES mIoU from 0.5624 to 0.5851 over raw boxes.

Surprising findings

• Grounding MLLMs (e.g., Ferret) perform well on grounding but poorly on quality-related objects, and IQA models score high on description quality yet low on grounded VQA.

Limitations

• GIQA-Bench is small (100 images, 250 QA items) and relies on expert annotation.
• Automated pipeline depends on Llama3, Grounding DINO, and Q-Instruct, so annotation errors propagate into the dataset.
• Discretized 20x20 grid coordinates reduce localization precision.

Experimental setup

Datasets: GIQA-160K Q-Pathway DQ-495K KonIQ-10K SPAQ LIVE-FB LIVE-itw AGIQA-3K ImageRewardDB KADIS-700K

Benchmarks: GIQA-Bench

Metrics: BLEU@4 LLM-Score mIoU Tag-Recall Accuracy (Yes/No) Accuracy (What/Which/How) Accuracy (Total)

Baselines: LLaVA-v1.5-7B LLaVA-v1.5-13B LLaVA-v1.6-7B mPLUG-Owl2-7B Shikra-7B Kosmos-2-1.6B Ferret-7B GroundingGPT-7B DepictQA-Wild-7B Q-Instruct

Builds on: Q-Instruct Grounding DINO LLaVA mPLUG-Owl2 Ferret

Closest papers by meaning

• IQA-Spider: Unifying Multi-Granularity Image Quality Assessment with Reasoning, Grounding and Referring (ICML 2026 · similarity 0.92)
• Revisiting MLLM Based Image Quality Assessment: Errors and Remedy (AAAI 2026 · similarity 0.91)
• Teaching Large Language Models to Regress Accurate Image Quality Scores Using Score Distribution (CVPR 2025 · similarity 0.89)
• GeM-VG: Towards Generalized Multi-image Visual Grounding with Multimodal Large Language Models (AAAI 2026 · similarity 0.89)
• Adaptive Image Quality Assessment via Teaching Large Multimodal Model to Compare (NeurIPS 2024 · similarity 0.88)
• Q-Bench: A Benchmark for General-Purpose Foundation Models on Low-level Vision (ICLR 2024 · similarity 0.88)