Research map / Image generation / Autoregressive and latent generation
Teaching Large Language Models to Regress Accurate Image Quality Scores Using Score Distribution
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DeQA-Score, an MLLM-based image quality scorer that trains on discretized Gaussian score distributions as soft labels plus a Thurstone-based fidelity loss for multi-dataset co-training.
Problem
MLLMs generate discrete tokens while image quality scores are continuous, and one-hot discretization of mean scores loses information and inter-image relationships; multi-dataset IQA co-training is also hampered by dataset distribution variation.
Approach
Discretize the Gaussian score distribution (not just the mean) into a soft label over five rating levels with a linear post-adjustment so the probabilities sum to one and match the mean, train with KL loss on the level token, and add a Thurstone fidelity loss on predicted score distributions to capture intra-dataset rankings.
Key results
- Single-dataset training on KonIQ: DeQA-Score reaches 0.953 PLCC on KonIQ vs 0.941 for Q-Align, and 0.892 vs 0.853 PLCC on LIVE-Wild.
- Co-training on KonIQ, SPAQ, KADID, PIPAL: DeQA-Score improves PLCC on PIPAL to 0.724 vs 0.702 for Q-Align, and on CSIQ to 0.863 vs 0.838.
- Score distribution prediction (Table 5): JS divergence on KonIQ drops from 0.415 (Q-Align) to 0.014 (DeQA-Score).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace one-hot score labels with distribution-based soft labels (ablation) | DeQA-Score · KonIQ | PLCC: 0.945 -> 0.954 | Helped |
| add fidelity loss to distribution-based soft-label training (ablation) | DeQA-Score · KonIQ | PLCC: 0.954 -> 0.957 | Helped |
| add discretized uncertainty to one-hot-label responses (ablation) | DeQA-Score · KonIQ | PLCC: 0.945 -> 0.935 | Hurt |
| add binary fidelity loss to one-hot-label training (ablation) | DeQA-Score · KonIQ | PLCC: 0.945 -> 0.944 | No clear effect |
| use randomly ordered quality-level tokens instead of common level tokens (ablation) | DeQA-Score · KonIQ | PLCC: 0.957 -> 0.942 | Hurt |
| reverse the order of common quality-level tokens (ablation) | DeQA-Score · KonIQ | PLCC: 0.957 -> 0.943 | Hurt |
| use number words instead of common quality-level tokens (ablation) | DeQA-Score · KonIQ | PLCC: 0.957 -> 0.945 | Hurt |
| use vehicle words instead of common quality-level tokens (ablation) | DeQA-Score · KonIQ | PLCC: 0.957 -> 0.953 | Hurt |
| use mixed unrelated words instead of common quality-level tokens (ablation) | DeQA-Score · KonIQ | PLCC: 0.957 -> 0.947 | Hurt |
| train DeQA-Score rather than Q-Align using single-dataset KonIQ training | mPLUG-Owl2-based IQA scoring · KonIQ | PLCC: 0.941 -> 0.953 | Helped |
| train DeQA-Score rather than Q-Align using single-dataset KonIQ training | mPLUG-Owl2-based IQA scoring · LIVE-Wild | PLCC: 0.853 -> 0.892 | Helped |
| add KADID to KonIQ and SPAQ co-training | DeQA-Score · TID2013 | SRCC: 0.616 -> 0.820 | Helped |
Surprising findings
- Random, reversed, numeric, or irrelevant level words still let the model learn the new level relationships, though common level texts perform best.
- PIPAL performance remains low (around 0.46 PLCC) for single-dataset KonIQ training, showing distortion gaps dominate generalization.
Limitations
- PIPAL (model-processed distortions) remains poorly predicted, with PLCC around 0.46-0.72 even after co-training.
- Adding PIPAL to co-training reduces performance on several other datasets such as CSIQ.
Experimental setup
Datasets: KonIQ SPAQ KADID PIPAL LIVE-Wild FLIVE TID2013 CSIQ AGIQA-3K
Benchmarks: KonIQ SPAQ KADID PIPAL LIVE-Wild AGIQA-3K TID2013 CSIQ FLIVE
Metrics: PLCC SRCC KL divergence JS divergence Wasserstein distance
Baselines: Q-Align Compare2Score NIQE BRISQUE NIMA HyperIQA DBCNN MUSIQ CLIP-IQA+ ManIQA
Builds on: Q-Align mPLUG-Owl2 UNIQUE Thurstone's model
Closest papers by meaning
- Revisiting MLLM Based Image Quality Assessment: Errors and Remedy (AAAI 2026 · similarity 0.94)
- Grounding-IQA: Grounding Multimodal Language Model for Image Quality Assessment (ICLR 2026 · similarity 0.89)
- Adaptive Image Quality Assessment via Teaching Large Multimodal Model to Compare (NeurIPS 2024 · similarity 0.88)
- IQA-Spider: Unifying Multi-Granularity Image Quality Assessment with Reasoning, Grounding and Referring (ICML 2026 · similarity 0.87)
- Q-Instruct: Improving Low-level Visual Abilities for Multi-modality Foundation Models (CVPR 2024 · similarity 0.87)
- Q-Bench: A Benchmark for General-Purpose Foundation Models on Low-level Vision (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.