Research map / Video / Video segmentation and tracking
Modular Blind Video Quality Assessment
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a modular blind video quality model (base ViT-CLIP predictor plus spatial and temporal rectifiers) trained with modular dropout so the base predictor works standalone and the rectifiers handle resolution and frame-rate effects.
Problem
Deep BVQA models analyze aggressively subsampled (resized, cropped, temporally subsampled) videos and are therefore insensitive to the actual spatial resolution and frame rate, which affect perceived quality.
Approach
A base quality predictor on sparse downsampled key frames is rectified by a spatial rectifier (Laplacian pyramids at native resolution through ResNet-18) and a temporal rectifier (SlowFast fast pathway on chunks at native frame rate), each producing scale and shift parameters; random dropout of rectifiers during training makes the base predictor usable standalone.
Key results
- On BVI-SR, the spatially rectified qs reaches SRCC 0.837 / PLCC 0.891, versus 0.494 for NIQE and 0.831 for VSFA.
- On BVI-HFR, the temporally rectified qt achieves SRCC 0.620 / PLCC 0.782, above FAVER's 0.556 / 0.639.
- On LSVQ-test, the full model qst reaches SRCC 0.895 / PLCC 0.895, versus 0.888 for DOVER; the weighted average over seven small UGC datasets is 0.882 / 0.890 for qst versus 0.860 / 0.870 for DOVER.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the spatial rectifier (ablation) | Modular BVQA · BVI-SR | SRCC / PLCC: 0.448 / 0.554 -> 0.837 / 0.891 | Helped |
| Add the temporal rectifier (ablation) | Modular BVQA · BVI-HFR | SRCC / PLCC: 0.035 / 0.206 -> 0.620 / 0.782 | Helped |
| Add both spatial and temporal rectifiers (ablation) | Modular BVQA · LIVE-Livestream and ETRI | SRCC / PLCC: LIVE-Livestream: 0.641 / 0.678 -> 0.802 / 0.811; ETRI: 0.924 / 0.943 -> 0.933 / 0.950 | Helped |
| Add the temporal rectifier (ablation) | Modular BVQA · LIVE-VQC, LIVE-Qualcomm, and LBVD | SRCC / PLCC: LIVE-VQC: 0.737 / 0.786 -> 0.833 / 0.851; LIVE-Qualcomm: 0.759 / 0.775 -> 0.825 / 0.822; LBVD: 0.701 / 0.700 -> 0.887 / | Helped |
| Add the spatial rectifier (ablation) | Modular BVQA · LIVE-VQC and LBVD | SRCC / PLCC: LIVE-VQC: 0.737 / 0.786 -> 0.714 / 0.776; LBVD: 0.701 / 0.700 -> 0.678 / 0.683 | Hurt |
| Add both rectifiers (ablation) | Modular BVQA · Seven small-scale UGC datasets | SRCC / PLCC: weighted average: 0.806 / 0.822 -> 0.882 / 0.890 | Helped |
| Use ViT-B rather than ResNet-50 as base predictor backbone (ablation) | Base quality predictor · LSVQ-test | SRCC / PLCC: 0.870 / 0.870 -> 0.895 / 0.895 | Helped |
| Initialize the backbone with CLIP contrastive learning rather than ImageNet supervision (ablation) | Base quality predictor · LSVQ-test | SRCC / PLCC: 0.866 / 0.867 -> 0.895 / 0.895 | Helped |
| Use bilinear or nearest-neighbor rather than bicubic resizing (ablation) | Base predictor and rectifiers · LSVQ-test and LSVQ-1080p | SRCC / PLCC: NN: 0.893 / 0.893; Bilinear: 0.891 / 0.891; Bicubic: 0.895 / 0.895 | No clear effect |
| Use GRU or Transformer rather than MLP regressor (ablation) | Base quality predictor · LSVQ-test and LSVQ-1080p | SRCC / PLCC: MLP: 0.895 / 0.895; GRU: 0.894 / 0.895; Transformer: 0.892 / 0.891 | No clear effect |
| Use ℓ1 or ℓ2 loss rather than PLCC loss (ablation) | Modular BVQA · LSVQ-test and LSVQ-1080p | SRCC / PLCC: ℓ1: 0.882 / 0.883; ℓ2: 0.887 / 0.886; PLCC: 0.895 / 0.895 | Hurt |
| Use RN34 or RN50 rather than RN18 spatial rectifier (ablation) | Spatial rectifier · BVI-SR and Waterloo-IVC-4K | SRCC / PLCC: RN18: 0.859 / 0.904 and 0.850 / 0.870; RN34: 0.866 / 0.907 and 0.810 / 0.855; RN50: 0.861 / 0.911 and 0.809 / 0.846 | Mixed |
Surprising findings
- The temporal rectifier can slightly harm accuracy on spatial-resolution datasets, and the spatial rectifier is inferior to the base model on LIVE-VQC and LBVD.
- Simple MLP regressors outperform GRU and Transformer regressors, and a minimalistic base predictor performs near state of the art on CVD2014, suggesting an easy dataset problem in UGC benchmarks.
Limitations
- Routing of rectifiers is fixed and determined by expert insight into datasets rather than adaptive.
- Aggregation of rectifiers is handcrafted (geometric/arithmetic mean) rather than learned.
- Adding a new rectifier requires retraining or fine-tuning all parameters.
Experimental setup
Datasets: BVI-SR Waterloo-IVC-4K BVI-HFR LIVE-YT-HFR LIVE-Livestream ETRI CVD2014 LIVE-Qualcomm KoNViD-1K LIVE-VQC YouTube-UGC LBVD
Benchmarks: BVI-SR Waterloo-IVC-4K BVI-HFR LIVE-YT-HFR LIVE-Livestream ETRI CVD2014 LIVE-Qualcomm KoNViD-1K LIVE-VQC YouTube-UGC LBVD
Metrics: SRCC PLCC
Baselines: NIQE BRISQUE VIDEVAL TLVQM RAPIQUE FAVER VSFA BTURA Li22 SimpleVQA FastVQA DOVER
Builds on: CLIP SlowFast ResNet-18 PLCC loss FastVQA
Closest papers by meaning
- PTM-VQA: Efficient Video Quality Assessment Leveraging Diverse PreTrained Models from the Wild (CVPR 2024 · similarity 0.87)
- KVQ: Boosting Video Quality Assessment via Saliency-guided Local Perception (CVPR 2025 · similarity 0.85)
- Learned Scanpaths Aid Blind Panoramic Video Quality Assessment (CVPR 2024 · similarity 0.85)
- FineVQ: Fine-Grained User Generated Content Video Quality Assessment (CVPR 2025 · similarity 0.85)
- AIGV-Assessor: Benchmarking and Evaluating the Perceptual Quality of Text-to-Video Generation with LMM (CVPR 2025 · similarity 0.84)
- Distilling Spatially-Heterogeneous Distortion Perception for Blind Image Quality Assessment (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.