Research map / Video / Video segmentation and tracking
PMQ-VE: Progressive Multi-Frame Quantization for Video Enhancement
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PMQ-VE, a two-stage post-training quantization framework for Transformer-based multi-frame video enhancement (STVSR, VSR, VFI) that uses per-frame backtracking bound search and progressive multi-teacher distillation from FP and high-bit teachers.
Problem
Quantizing Transformer-based multi-frame video enhancement models to low bit-widths causes large performance and detail loss because activation ranges differ across frames and low-bit students struggle to learn from only a full-precision teacher.
Approach
Assign frame-specific clipping bounds found by percentile-initialized backtracking search (BMFQ), and train low-bit students with progressive distillation from both full-precision and intermediate-bit (8/4-bit) teachers (PMTD).
Key results
- On STVSR at 2/2-bit, Ours reaches 23.48 dB PSNR on Vid4 vs 22.91 for 2DQuant, and 30.33 dB on Vimeo-Fast vs 29.38.
- At 4/4-bit on EMA-VFI [T] Vimeo90K, Ours achieves 28.41 dB PSNR vs 28.06 for 2DQuant.
- Ablation on STVSR 2-bit: baseline 12.67 dB, per-frame quantization 19.64 dB, adding BMFQ 27.56 dB, adding PMTD 30.33 dB.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add per-frame quantization (ablation) | PMQ-VE on STVSR · STVSR ablation (2-bit) | PSNR: 12.67 -> 19.64 | Helped |
| Add BMFQ to per-frame quantization (ablation) | PMQ-VE on STVSR · STVSR ablation (2-bit) | PSNR: 19.64 -> 27.56 | Helped |
| Add PMTD to per-frame quantization and BMFQ (ablation) | PMQ-VE on STVSR · STVSR ablation (2-bit) | PSNR: 27.56 -> 30.33 | Helped |
| Use PMQ-VE instead of 2DQuant | RSTT-S · Vimeo-Fast (STVSR, 4/4-bit) | PSNR: 33.59 -> 34.69 | Helped |
| Use PMQ-VE instead of 2DQuant | EMA-VFI [D] · Vimeo90K (VFI, 4/4-bit) | PSNR: 28.78 -> 29.59 | Helped |
| Use PMQ-VE instead of 2DQuant | MIA · Vimeo90K (VSR, 4-bit) | PSNR: 36.92 -> 37.34 | Helped |
Limitations
- Evaluated only on Transformer-based video enhancement models; diffusion-based DiT methods untested.
- Gains over 2DQuant are modest on some settings (e.g., about 0.35 dB at 4-bit VFI).
Experimental setup
Datasets: Vimeo-90K
Benchmarks: Vid4 Vimeo-90K Vimeo-Fast Vimeo-Medium Vimeo-Slow
Metrics: PSNR SSIM LPIPS NIQE
Baselines: OpenVINO TensorRT SNPE Percentile MinMax NoisyQuant DBDC+Pac 2DQuant
Builds on: RSTT MIA EMA-VFI 2DQuant DBDC+Pac
Closest papers by meaning
- Q-VDiT: Towards Accurate Quantization and Distillation of Video-Generation Diffusion Transformers (ICML 2025 · similarity 0.87)
- ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generation (ICLR 2025 · similarity 0.87)
- DSAQuant: Denoising-Stage-Aligned Quantization-Aware Training for Video Generation (NeurIPS 2026 · similarity 0.87)
- Q-ARVD: Quantizing Autoregressive Video Diffusion Models (NeurIPS 2026 · similarity 0.86)
- $\text{S}^2$Q-VDiT: Accurate Quantized Video Diffusion Transformer with Salient Data and Sparse Token Distillation (NeurIPS 2025 · similarity 0.86)
- Q-DiT: Accurate Post-Training Quantization for Diffusion Transformers (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.