Research map / Video / Video segmentation and tracking
Perception-Oriented Video Frame Interpolation via Asymmetric Blending
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PerVFI, a perception-oriented video frame interpolation method that combines asymmetric synergistic blending with a self-learned sparse quasi-binary mask and a conditional normalizing flow generator trained with NLL plus perceptual loss.
Problem
Video frame interpolation suffers from blur and ghosting caused by inevitable motion estimation errors and temporal supervision misalignment, where reconstruction losses favor blurry averages.
Approach
Asymmetric blending takes primary content from one reference frame and complementary occlusion content from the other, controlled by an adaptive dilation quasi-binary mask; a conditional normalizing flow generator models the output distribution with NLL plus VGG perceptual loss instead of pixel L1.
Key results
- On DAVIS 1080P, PerVFI achieves VFIPS 72.52 and FloLPIPS 0.087 versus EMA-VFI's 60.87 and 0.131, though its PSNR (26.23) is lower than EMA-VFI's 27.61.
- On Xiph 2K, PerVFI reaches LPIPS 0.038 and VFIPS 68.67, the best among compared methods.
- In a user study with 33 participants over 98 videos, PerVFI was preferred over compared methods (e.g., 85.7% vs EMA-VFI in the reported A/B split).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PerVFI instead of RIFE | Video frame interpolation · DAVIS-2017 480P | LPIPS: 0.085 -> 0.077 | Helped |
| Use PerVFI instead of AMT | Video frame interpolation · DAVIS-2017 480P | VFIPS: 80.57 -> 87.51 | Helped |
| Use PerVFI instead of RIFE | Video frame interpolation · DAVIS-2017 1080P | VFIPS: 62.56 -> 72.52 | Helped |
| Use PerVFI instead of RIFE | Video frame interpolation · Xiph 2K | FloLPIPS: 0.050 -> 0.032 | Helped |
| Use PerVFI instead of AMT | Video frame interpolation · Xiph 2K | VFIPS: 65.60 -> 68.67 | Helped |
| Use PerVFI instead of RIFE | Video frame interpolation · Xiph 4K | VFIPS: 54.31 -> 57.47 | Helped |
| Add the quasi-binary mask instead of a binary mask (ablation) | PerVFI · DAVIS-2017 480P | VFIPS: 81.25 -> 83.30 | Helped |
| Add random noise during mask training (ablation) | PerVFI · DAVIS-2017 480P | VFIPS: 81.24 -> 83.30 | Helped |
| Add optical-flow prior information to PAM (ablation) | PerVFI · DAVIS-2017 480P | VFIPS: 82.27 -> 83.30 | Helped |
| Use the bidirectional loss instead of L1 loss (ablation) | PerVFI · DAVIS-2017 480P | VFIPS: 80.50 -> 83.30 | Helped |
| Add perceptual loss to NLL loss (ablation) | PerVFI · DAVIS-2017 480P | VFIPS: 81.88 -> 83.30 | Helped |
| Use a quasi-binary mask instead of a fully adaptive mask (ablation) | PerVFI · DAVIS-2017 480P | VFIPS: 78.20 -> 83.30 | Helped |
Surprising findings
- Symmetric blending with a fully adaptive mask gets higher PSNR but lower VFIPS, showing PSNR is not aligned with perceptual quality.
- STMFNet achieves the highest PSNR and SSIM on several benchmarks yet lags on perceptual metrics.
Limitations
- PSNR and SSIM are lower than several baselines such as EMA-VFI and STMFNet on some benchmarks (e.g., PerVFI PSNR 26.83 vs EMA-VFI 28.83 on DAVIS 480P).
- Performance depends on motion estimator choice (RAFT or GMFlow) and on a temperature hyperparameter for latent sampling trading off metrics.
Experimental setup
Datasets: Vimeo-90K
Benchmarks: DAVIS-2017 Xiph Vimeo-90K
Metrics: PSNR SSIM LPIPS DISTS VFIPS FloLPIPS
Baselines: EDSC RIFE VFIFormer EMA-VFI AMT STMFNet LDMVFI
Builds on: RAFT conditional normalizing flow (Glow-style) softmax splatting deformable convolution
Closest papers by meaning
- Disentangled Motion Modeling for Video Frame Interpolation (AAAI 2025 · similarity 0.90)
- Exploring Motion Ambiguity and Alignment for High-Quality Video Frame Interpolation (CVPR 2023 · similarity 0.89)
- Sparse Global Matching for Video Frame Interpolation with Large Motion (CVPR 2024 · similarity 0.88)
- LDMVFI: Video Frame Interpolation with Latent Diffusion Models (AAAI 2024 · similarity 0.87)
- Explicit Depth-Aware Blurry Video Frame Interpolation Guided by Differential Curves (CVPR 2025 · similarity 0.87)
- Repurposing Pre-trained Video Diffusion Models for Event-based Video Interpolation (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.