Research map / Video / Video segmentation and tracking
Multi-Frame Deformable Look-Up Table for Compressed Video Quality Enhancement
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a multi-frame deformable LUT framework for low-latency compressed video quality enhancement, where a CNN predicts multi-scale deformable offsets from frames and motion vectors, and the alignment, temporal feature and fusion modules are converted to 4-bit LUTs at inference.
Problem
Compressed video quality enhancement (CVQE) with DNNs is computationally expensive, limiting use in low-latency settings, while existing LUT-based methods are single-frame and not tailored to CVQE.
Approach
Combine a lightweight motion-vector-modulated CNN that predicts multi-scale deformable convolution offsets with LUT-converted temporal feature extraction and multi-scale fusion modules, so that alignment and reconstruction run as table lookups with tetrahedral interpolation at inference.
Key results
- Average ΔPSNR of 0.44 dB at QP37 versus 0.15 dB for RC-LUT and 0.65 dB for MFQE 2.0 on the 18 test sequences.
- Runs at 36 to 83.6 fps depending on resolution/stage; about 36.1 fps at 1080p in the LUT-converted model, reported as real-time (30 fps) at 1080p.
- Ablations show multi-frame, multi-scale TFEM and MSFM, and MV modulation each improve ΔPSNR (0.33 to 0.44 dB at QP37).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the full multi-frame deformable LUT method instead of LUT baselines | CVQE on JCT-VC 18 test videos at QP37 · JCT-VC 18 test videos, QP37 average | ΔPSNR (dB): MuLUT 0.13; RC-LUT 0.15; Ours 0.44 | Helped |
| Convert the feature-extraction and fusion convolutions to LUTs | Proposed CVQE method · JCT-VC QP37, 1080p | runtime (fps): 3.7 -> 36.1 | Helped |
| Use LUT-Convert finetuning rather than sampling finetuning | Proposed CVQE method · JCT-VC QP37, 1080p | ΔPSNR (dB): 0.446 -> 0.439 | Hurt |
| Use multi-frame information rather than a single-frame structure (ablation) | Proposed CVQE method · JCT-VC QP37 | ΔPSNR/ΔSSIM: 0.33/0.76 -> 0.44/1.01 | Helped |
| Use multi-scale TFEM rather than single-scale TFEM (ablation) | Proposed CVQE method · JCT-VC QP37 | ΔPSNR/ΔSSIM: 0.36/0.83 -> 0.38/0.89 | Helped |
| Enhance the multi-scale aligned information (ablation) | Proposed CVQE method · JCT-VC QP37 | ΔPSNR/ΔSSIM: 0.38/0.89 -> 0.41/0.98 | Helped |
| Introduce motion vectors for feature modulation (ablation) | Proposed CVQE method · JCT-VC QP37 | ΔPSNR/ΔSSIM: 0.41/0.98 -> 0.44/1.01 | Helped |
Limitations
- Accuracy trails high-latency CNN methods such as MFQE 2.0 (0.44 vs 0.65 dB average ΔPSNR at QP37).
- Converting float networks to integer LUTs causes a small performance drop, requiring extra finetuning stages.
- Evaluated only on luma (Y channel) and a single codec setting (HEVC LDP).
Experimental setup
Datasets: CVCP HEVC standard test sequences
Benchmarks: 18 HEVC JCT-VC test sequences
Metrics: ΔPSNR ΔSSIM runtime (fps) FLOPs
Baselines: MFQE 2.0 STDF-R1 MFQR AR-CNN MuLUT RC-LUT SR-LUT
Builds on: MuLUT RC-LUT deformable convolution MFQE 2.0 ConvLUT
Closest papers by meaning
- RivuletMLP: An MLP-based Architecture for Efficient Compressed Video Quality Enhancement (CVPR 2025 · similarity 0.85)
- PMQ-VE: Progressive Multi-Frame Quantization for Video Enhancement (NeurIPS 2025 · similarity 0.84)
- CPGA: Coding Priors-Guided Aggregation Network for Compressed Video Quality Enhancement (CVPR 2024 · similarity 0.83)
- Modular Blind Video Quality Assessment (CVPR 2024 · similarity 0.82)
- Look-Up Table Compression for Efficient Image Restoration (CVPR 2024 · similarity 0.82)
- KVQ: Boosting Video Quality Assessment via Saliency-guided Local Perception (CVPR 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.