Research map / Image generation / Autoregressive and latent generation

Multidimensional Observer Model and Perceptual Dimensions of Human Image Quality Assessment

NeurIPS 2026 · Image generation › Autoregressive and latent generation · approach: Learning-based image quality assessment · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a multi-dimensional Gaussian observer model over CORnet-S ventral-stream features that fits triplet 2AFC IQA data and finds human quality judgments live in very low-dimensional perceptual spaces, with higher dimensionality for semantic (NIGHTS) than low-level (BAPPS, PieAPP) judgments.

Problem

Image quality assessment metrics are optimized for prediction accuracy rather than for revealing the perceptual space humans use for quality judgments, including its dimensionality and structure across low- and high-level tasks.

Approach

Represents each image as a Gaussian distribution in a latent perceptual space that is a learned affine projection of frozen ventral-stream (CORnet-S) features, with input-dependent covariance via a Cholesky-parameterized MLP; choices are modeled by Monte Carlo comparisons of noisy triplet distances, fit by maximum likelihood, and the saturating dimensionality is read off as an intrinsic perceptual

Key results

• On BAPPS the observer model reaches 0.688 agreement, the best among compared metrics; on PieAPP it trails the best (0.710 vs 0.719 DreamSim) and on NIGHTS it is 0.859 vs DreamSim's 0.957.
• Saturating dimensionality is about 3-8 for low-level datasets BAPPS and PieAPP but reaches 96 at IT for the high-level NIGHTS dataset.
• A frequency-space DCT control at matched sparsity is about 0.1 lower in agreement, and matching N=6 performance requires roughly two orders of magnitude more DCT dimensions.

Surprising findings

• Including IT features degrades performance on low-level tasks, while the contrast pyramid (retinal) encoding gives the lowest performance on all datasets.
• The observer model does not beat DreamSim on NIGHTS, where DreamSim's foundation-model features reach 0.957 accuracy.

Limitations

• Mean and covariance are only identifiable within a triplet, so distributions cannot be compared across non-shared triplets.
• The model does not set a new state of the art in 2AFC prediction and is not evaluated on rating datasets in the main text.

Experimental setup

Datasets: BAPPS PieAPP NIGHTS TID2013 KADID-10k CSIQ LIVE Release 2

Benchmarks: BAPPS PieAPP NIGHTS

Metrics: agreement score KL divergence accuracy (1-BER) saturation point n*

Closest papers by meaning

• When does perceptual alignment benefit vision representations? (NeurIPS 2024 · similarity 0.87)
• FoMo: Forking Moment in Generative Trajectory as a Perceptual Distance (NeurIPS 2026 · similarity 0.87)
• Perceptual Scales Predicted by Fisher Information Metrics (ICLR 2024 · similarity 0.87)
• Evaluating Multiview Object Consistency in Humans and Image Models (NeurIPS 2024 · similarity 0.87)
• Evaluation of Visual Processing Should Be Human-Centered, Not Metric-Centered (NeurIPS 2026 · similarity 0.86)
• Synthesizing Images on Perceptual Boundaries of ANNs for Uncovering and Manipulating Human Perceptual Variability (ICML 2025 · similarity 0.86)