Research map / Video / Video segmentation and tracking
BiM-VFI: Bidirectional Motion Field-Guided Frame Interpolation for Video with Non-uniform Motions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BiM-VFI, a video frame interpolation model that uses a bidirectional motion field (magnitude ratio and angle of bidirectional flows) as a training-time motion descriptor, plus a BiM-guided flow net, content-aware upsampling, and teacher-student flow distillation, to produce sharper frames under non-uniform motion.
Problem
Video frame interpolation trained on non-uniform motion suffers time-to-location ambiguity, so models learn averaged trajectories and produce blurred intermediate frames.
Approach
Describe pixel motion with a Bidirectional Motion Field (magnitude ratio plus angle between flows from the target frame to each source frame), which resolves both velocity and directional ambiguity during training; the teacher process uses the ground-truth target frame to produce VFI-centric flows and BiMs that supervise the student, and at inference a uniform-motion BiM is used.
Key results
- On SNU-FILM-arb Extreme, BiM-VFI reports LPIPS 0.070 and STLPIPS 0.034 versus 0.089/0.055-type gaps for the strongest prior baselines, supporting the abstract's claimed 26% LPIPS and 45% STLPIPS improvements.
- Ablation on Vimeo90K septuplet: BiM with magnitude ratio and angle (LPIPS 0.070, STLPIPS 0.043) beats time indexing (0.098/0.077) and removing angle (0.074/0.045).
- BiM-VFI underperforms on pixel metrics (e.g., PSNR 26.83 vs 29.41 for EMA-VFI on Vimeo90K septuplet) because it assumes uniform motion at inference.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use BiM instead of time indexing T (ablation) | BiM-VFI · Vimeo90K-septuplet | LPIPS: 0.098 -> 0.070 | Helped |
| Add the angle component Φ to the motion ratio R (ablation) | BiM-VFI · Vimeo90K-septuplet | LPIPS | Helped |
| Use BiM motion ratio R instead of time indexing T alongside Φ (ablation) | BiM-VFI · Vimeo90K-septuplet | LPIPS: 0.074 -> 0.070 | Helped |
| Add KDVCF flow supervision (ablation) | BiM-VFI · Vimeo90K-septuplet | LPIPS: 0.076 -> 0.070 | Helped |
| Use KDVCF instead of FlowFormer-extracted flow supervision (ablation) | BiM-VFI · Vimeo90K-septuplet | LPIPS: 0.076 -> 0.070 | Helped |
| Use elementwise multiplication instead of concatenation and convolution in BiM-MConv (ablation) | BiM-VFI · Vimeo90K-septuplet | LPIPS: 0.071 -> 0.070 | Helped |
| Add CAUN adaptive upsampling (ablation) | BiM-VFI · Vimeo90K-septuplet | LPIPS: 0.076 -> 0.070 | Helped |
Surprising findings
- Using only the magnitude ratio (time-indexing-like) training failed to train at all in the R-only ablation.
- Pixel-centric metrics (PSNR/SSIM) are misleading here: the method scores lower PSNR yet is perceptually better because baselines produce averaged blurry frames.
Limitations
- Assumes uniform motion at inference because the target BiM is unavailable, so pixel metrics lag and interpolated motion may not match non-uniform ground truth.
- Pixel-centric metrics are weaker than baselines on Vimeo90K septuplet.
Experimental setup
Datasets: Vimeo90K triplet Vimeo90K septuplet SNU-FILM SNU-FILM-arb XTest
Benchmarks: Vimeo90K septuplet SNU-FILM-arb Vimeo90K triplet SNU-FILM XTest
Metrics: PSNR SSIM LPIPS STLPIPS NIQE
Baselines: RIFE IFRNet M2M-PWC AMT-S UPRNet EMA-VFI [D,R]-EMA-VFI
Builds on: IFRNet EMA-VFI distance indexing RAFT-style convex upsampling Softmax splatting-free backward warping
Closest papers by meaning
- IQ-VFI: Implicit Quadratic Motion Estimation for Video Frame Interpolation (CVPR 2024 · similarity 0.89)
- Generalizable Implicit Motion Modeling for Video Frame Interpolation (NeurIPS 2024 · similarity 0.89)
- Disentangled Motion Modeling for Video Frame Interpolation (AAAI 2025 · similarity 0.88)
- Explicit Depth-Aware Blurry Video Frame Interpolation Guided by Differential Curves (CVPR 2025 · similarity 0.88)
- Sparse Global Matching for Video Frame Interpolation with Large Motion (CVPR 2024 · similarity 0.88)
- VFIMamba: Video Frame Interpolation with State Space Models (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.