Research map / Video / Video segmentation and tracking
Learned Scanpaths Aid Blind Panoramic Video Quality Assessment
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes an end-to-end optimized blind panoramic video quality assessment method that pairs a learned probabilistic GMM scanpath generator (trained by minimizing expected code length) with a planar VQA model over differentiable viewport sequences.
Problem
Blind panoramic video quality assessment (PVQA) must account for diverse, uncertain user viewing behaviors (scanpaths) and the spherical structure, but existing scanpath methods rely on human scanpaths or optimize the scanpath generator separately from the quality predictor.
Approach
A probabilistic scanpath generator (GMM over relative uv coordinates, trained by expected code length) is made differentiable via Gumbel-softmax and reparameterization, so it is jointly fine-tuned with any planar VQA model; panoramic images are handled as videos of identical frames.
Key results
- On VRVQW, the Assessor360-based model reaches SRCC 0.822 / PLCC 0.823 in-dataset, versus 0.779 / 0.781 for ScanpathVQA.
- On OIQA, the ScanpathVQA-based variant reaches SRCC 0.915 versus 0.945 for GSR-X, while the Assessor360-based variant reaches 0.960 SRCC.
- The learned scanpath generator outperforms Li23 and other baselines in explaining human scores (CVIQD SRCC 0.928 vs 0.814) and in minOD (0.536 vs 0.542 for Li23).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use learned scanpaths with the Assessor360 quality assessor | Assessor360 · VRVQW | SRCC: 0.779 -> 0.822 | Helped |
| Use learned scanpaths with the Assessor360 quality assessor | Assessor360 · CVIQD | SRCC: 0.955 -> 0.972 | Helped |
| Use learned scanpaths with the Assessor360 quality assessor | Assessor360 · OIQA | SRCC: 0.946 -> 0.960 | Helped |
| Use learned scanpaths with Assessor360 | Assessor360 · OIQA → CVIQD | SRCC: 0.859 -> 0.872 | Helped |
| Use learned scanpaths with Assessor360 | Assessor360 · CVIQD → OIQA | SRCC: 0.338 -> 0.703 | Helped |
| Vary scanpaths between training epochs instead of fixing them (ablation) | Two-stage ScanpathVQA quality prediction · CVIQD | SRCC: 0.710 -> 0.830 | Helped |
| End-to-end optimize the scanpath generator and quality assessor (ablation) | Three-stage ScanpathVQA quality prediction · CVIQD | SRCC: 0.830 -> 0.928 | Helped |
| End-to-end optimize the scanpath generator and quality assessor (ablation) | Three-stage ScanpathVQA quality prediction · OIQA | SRCC: 0.798 -> 0.914 | Helped |
| Add visual context to the scanpath generator (ablation) | Blind PVQA · VRVQW | SRCC: 0.805 -> 0.816 | Helped |
| Add visual context to the scanpath generator (ablation) | Blind PVQA · CVIQD | SRCC: 0.928 -> 0.937 | Helped |
| Add visual context to the scanpath generator (ablation) | Blind PVQA · OIQA | SRCC: 0.914 -> 0.925 | Helped |
| End-to-end optimize scanpath generation rather than optimizing it independently (ablation) | Scanpath generator · VRVQW | maxTC: 0.781 -> 0.805 | Helped |
Surprising findings
- Adding historical video frames as visual context to the scanpath generator yields negligible gains (about 0.01 SRCC) at 27M vs 1M parameters, so visual context was omitted.
- Assessor360 drops sharply in cross-dataset testing (CVIQD to OIQA SRCC 0.338), and the proposed scanpath generator restores performance (0.703).
Limitations
- Visual context in the scanpath generator was found to add little benefit, so the generator ignores video content.
- Viewport sequence number and length tuned only for ScanpathVQA and may not transfer to other assessors.
Experimental setup
Datasets: VRVQW CVIQD OIQA
Benchmarks: VRVQW CVIQD OIQA
Metrics: SRCC PLCC minOD maxTC
Baselines: NIQE MC360IQA Wen24 ScanpathVQA GSR-S GSR-X Assessor360
Builds on: expected code length scanpath modeling (Li et al. 2023) ScanpathVQA Assessor360 GSR Gumbel-softmax
Closest papers by meaning
- Modular Blind Video Quality Assessment (CVPR 2024 · similarity 0.85)
- ViewPoint: Panoramic Video Generation with Pretrained Diffusion Models (NeurIPS 2025 · similarity 0.84)
- Distilling Spatially-Heterogeneous Distortion Perception for Blind Image Quality Assessment (CVPR 2025 · similarity 0.84)
- Integrating Global Context Contrast and Local Sensitivity for Blind Image Quality Assessment (ICML 2024 · similarity 0.83)
- PanoWan: Lifting Diffusion Video Generation Models to 360$^\circ$ with Latitude/Longitude-aware Mechanisms (NeurIPS 2025 · similarity 0.83)
- KVQ: Boosting Video Quality Assessment via Saliency-guided Local Perception (CVPR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.