Research map / Video / Video segmentation and tracking
PTM-VQA: Efficient Video Quality Assessment Leveraging Diverse PreTrained Models from the Wild
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PTM-VQA, which extracts frozen features from several diverse pretrained vision models, fuses them with DBI-based weights, and trains a light head with an ICID metric-learning loss for no-reference video quality assessment.
Problem
No-reference video quality assessment is limited by small MOS-annotated datasets, and fine-tuning many large pretrained models is computationally costly; it is unclear which pretrained models help VQA and how to combine them.
Approach
Use frozen features from diverse pretrained models (image, video, CLIP) transformed to a common dimension, integrate them with weights from Davies-Bouldin Index clustering quality, and impose intra-model consistency and inter-cluster divisibility constraints (ICID loss) built on pseudo-clusters from MOS intervals.
Key results
- PTM-VQA reaches PLCC 0.8718 / SRCC 0.8568 on KoNViD-1k, competitive with VQT (0.8633 mean) and above BVQA and STDAM (which use extra IQA data).
- On LSVQ test, PTM-VQA-VQC obtains PLCC 0.8637 / SRCC 0.8545, versus PVQ (w patch) at 0.828 / 0.827.
- Ablation on KoNViD-1k shows removing the ICID constraints drops PLCC from 0.8718 to 0.7968 (no L_intra) or 0.7867 (no L_inter); training takes about two hours on one GPU.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PTM-VQA instead of VQT | KoNViD-1k · KoNViD-1k | PLCC: 0.8684 -> 0.8718 | Helped |
| Use PTM-VQA instead of VQT | KoNViD-1k · KoNViD-1k | SRCC: 0.8582 -> 0.8568 | Hurt |
| Use PTM-VQA instead of BVQA | LIVE-VQC · LIVE-VQC | PLCC: 0.8415 -> 0.8198 | Hurt |
| Use PTM-VQA instead of BVQA | LIVE-VQC · LIVE-VQC | SRCC: 0.8412 -> 0.8110 | Hurt |
| Use PTM-VQA instead of VQT | YouTube-UGC · YouTube-UGC | PLCC: 0.8514 -> 0.8570 | Helped |
| Use PTM-VQA instead of VQT | YouTube-UGC · YouTube-UGC | SRCC: 0.8357 -> 0.8578 | Helped |
| Use 6 MOS clusters instead of 2 (ablation) | PTM-VQA · KoNViD-1k | PLCC: 0.8277 -> 0.8718 | Helped |
| Use 6 MOS clusters instead of 4 (ablation) | PTM-VQA · KoNViD-1k | SRCC: 0.8012 -> 0.8568 | Helped |
| Replace DBI weighting with equal model weights (ablation) | PTM-VQA · KoNViD-1k | PLCC: 0.8631 -> 0.8718 | Hurt |
| Replace DBI weighting with equal model weights (ablation) | PTM-VQA · LIVE-VQC | SRCC: 0.8107 -> 0.8110 | No clear effect |
| Replace DBI weighting with equal model weights (ablation) | PTM-VQA · YouTube-UGC | SRCC: 0.8446 -> 0.8578 | Helped |
| Use the LIVE-VQC pretrained-model combination (ablation) | PTM-VQA · LSVQ-Test | PLCC: 0.8536 -> 0.8637 | Mixed |
Surprising findings
- Models pretrained on quality-unrelated tasks (e.g., CLIP, action recognition) yield clusters aligned with MOS, suggesting quality-relevant knowledge emerges without quality labels.
- Frozen-feature PTM-VQA matches or beats several fine-tuning methods that use extra IQA data, such as BVQA and STDAM, on KoNViD-1k.
Limitations
- Relies on MOS-derived pseudo-clusters and requires cluster-balanced batches, with few samples at the MOS endpoints.
- Model combinations are chosen per dataset, and the random-selection baseline shows high variance, so selection depends on DBI computed on the target dataset.
Experimental setup
Datasets: KoNViD-1k LIVE-VQC YouTube-UGC LSVQ
Benchmarks: KoNViD-1k LIVE-VQC YouTube-UGC LSVQ
Metrics: PLCC SRCC mean of PLCC and SRCC
Baselines: VSFA TLVQM RAPIQUE StarVQA BVQA STDAM Fast-VQA VQT BRISQUE NIQE VIIDEO RIRNet
Builds on: triplet loss BVQA LMNN Davies-Bouldin Index
Closest papers by meaning
- Knowledge Guided Semi-supervised Learning for Quality Assessment of User Generated Videos (AAAI 2024 · similarity 0.88)
- AIGV-Assessor: Benchmarking and Evaluating the Perceptual Quality of Text-to-Video Generation with LMM (CVPR 2025 · similarity 0.88)
- Modular Blind Video Quality Assessment (CVPR 2024 · similarity 0.87)
- KVQ: Boosting Video Quality Assessment via Saliency-guided Local Perception (CVPR 2025 · similarity 0.87)
- FineVQ: Fine-Grained User Generated Content Video Quality Assessment (CVPR 2025 · similarity 0.85)
- Q-Bench-Video: Benchmark the Video Quality Understanding of LMMs (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.