Research map / Vision-language and multimodal / Human and embodied modeling
Image Quality Assessment: From Human to Machine Preference
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Image Quality Assessment for Machine Vision and builds the Machine Preference Database (MPD), with 30k reference/distorted pairs annotated by 15 LMMs and 15 CV models across 7 downstream tasks, then shows human-centric IQA metrics fail to predict machine preference.
Problem
Existing image quality assessment is built around human subjective preference, but machines now consume most visual data and their quality preferences depend on downstream tasks; no database annotates overall machine preference scores.
Approach
Defines machine subjective preference through downstream task outputs (LMM yes/no, MCQ, VQA, captioning and CV segmentation, detection, retrieval) compared between reference and distorted images, aggregated as a machine MOS, and uses this to benchmark IQA metrics trained on machine labels.
Key results
- Across 12 IQA metrics trained on MPD, SRCC on Severe Distortion reaches roughly 0.8 at best (e.g., ARNIQA 0.8337 on NSI), but on Mild Distortion the best SRCC is about 0.5 (HyperIQA 0.5388 on NSI), and no metric clearly beats the PSNR baseline.
- Human inter-subject SRCC is 0.76 versus 0.62 among machine subjects, indicating greater divergence in machine preferences.
- Metrics trained on KADID-10K or TID2013 score below 0.4 SRCC on MPD (Mild/Severe), while metrics trained on MPD reach about 0.7-0.8 SRCC on human datasets.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use human-centric IQA metrics to predict machine preference | MPD · MPD | SRCC | Mixed |
| Train IQA metrics on machine preference data rather than human-centric data | IQA metrics · MPD mild and severe validation | SRCC: 0.1514; 0.4753 | Helped |
| Train IQA metrics on machine preference data | IQA metrics · KADID-10K and TID2013 validation | SRCC: 0.7 to 0.8 | No clear effect |
Surprising findings
- The more accurately an IQA method models the human visual system, the harder it is to predict machine preference.
- Models trained on human-preference databases fail badly on machine preference, while machine-trained models still predict human preference reasonably well.
Limitations
- Machine preference labels depend on the specific set of LMMs and CV models used, which may not capture future models.
- Mild-distortion predictions remain poor, so current IQA metrics do not characterize machine preference well.
Experimental setup
Datasets: MPD KADID-10K TID2013
Benchmarks: MPD
Metrics: SRCC KRCC PLCC
Baselines: PSNR SSIM AHIQ CKDN DISTS LPIPS PieAPP TOPIQ-FR ARNIQA CLIPIQA DBCNN HyperIQA
Builds on: TOPIQ pyiqa CLIP
Closest papers by meaning
- AI-generated Image Quality Assessment in Visual Communication (AAAI 2025 · similarity 0.86)
- Adaptive Image Quality Assessment via Teaching Large Multimodal Model to Compare (NeurIPS 2024 · similarity 0.86)
- Toward Generalized Image Quality Assessment: Relaxing the Perfect Reference Quality Assumption (CVPR 2025 · similarity 0.86)
- Evaluation of Visual Processing Should Be Human-Centered, Not Metric-Centered (NeurIPS 2026 · similarity 0.85)
- Causal-IQA: Towards the Generalization of Image Quality Assessment Based on Causal Inference (ICML 2024 · similarity 0.85)
- QMamba: On First Exploration of Vision Mamba for Image Quality Assessment (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.