Research map / Video / Video segmentation and tracking
Sparse Global Matching for Video Frame Interpolation with Large Motion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SGM-VFI, a video frame interpolation pipeline that refines initial intermediate flows with sparse global matching (GMFlow features) at error-prone points located by a difference map, and evaluates on curated large-motion subsets of X-Test, Xiph and SNU-FILM.
Problem
Video frame interpolation struggles with large motion because local receptive fields of flow-based models cannot establish correspondences for large pixel displacements.
Approach
Locate flaws in initial intermediate flows with a difference map computed by bidirectional warping, then perform sparse global matching (GMFlow global features) only at the top-k flawed points, shift the correspondences into intermediate flow compensation, and adaptively merge them with the local flow via a convex-combination flow merge block.
Key results
- On X-Test-L-4K, the 1/2-points variant reaches 29.91 dB PSNR, versus 29.02 for XVFI and 29.17 for... (BiFormer 30.11 at 2K; Ours-1/2-Points 30.99 at 2K)
- Sparse global matching improves X-Test-L-2K PSNR from 30.58 (local branch fine-tuned) to 30.99 with 1/2 points, a gain of about 0.4 dB.
- Ablation: top-k difference-map selection outperforms random sampling at every sparsity level (e.g., 30.83 vs 30.55 PSNR at 1/8 points on X-Test-L-2K).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add sparse global matching with 1/8-point flow compensation (ablation) | Ours-local-branch · X-Test-L | PSNR: 30.39 -> 30.83; 29.25 -> 29.73 | Helped |
| Add sparse global matching with 1/2-point flow compensation (ablation) | Ours-local-branch · X-Test-L | PSNR: 30.58 -> 30.99; 29.44 -> 29.91 | Helped |
| Increase the sparse matching fraction from 1/8 to 1/2 (ablation) | Ours · X-Test-L | PSNR: 30.83 -> 30.99; 29.73 -> 29.91 | Helped |
| Fine-tune the local branch on X-Train without global matching (ablation) | Ours local branch · X-Test-L | PSNR: 30.39 -> 30.58; 29.25 -> 29.44 | Helped |
| Use full rather than sparse global matching (ablation) | Ours · X-Test-L | PSNR: 30.99 -> 31.03; 29.91 -> 29.95 | Helped |
| Select top-k defective points using the difference map rather than random points (ablation) | Ours sparse global matching · X-Test-L | PSNR: 30.89 -> 30.99; 30.67 -> 30.88; 30.55 -> 30.83 | Helped |
| Remove the flow merge block (ablation) | Ours sparse global matching · X-Test-L | PSNR: 30.75 -> 30.99; 30.77 -> 30.88; 30.75 -> 30.83 | Hurt |
| Add sparse global matching to the local branch (ablation) | Ours · SNU-FILM easy | PSNR: 40.15 -> 40.15 | No clear effect |
| Add sparse global matching to the local branch (ablation) | Ours · SNU-FILM medium | PSNR: 36.07 -> 36.05 | Hurt |
Surprising findings
- Random point selection for global matching can decrease accuracy relative to the local-only fine-tuned model at sparse ratios, by replacing correct flow with inaccurate compensation.
- Performance saturates as more points receive global matching, since not every point needs a global receptive field.
Limitations
- Curated large-motion subsets are defined by the authors and differ from standard benchmarks.
- Performance on small-to-medium motion is roughly comparable to VFIFormer rather than clearly superior.
- Relies on a pretrained frozen GMFlow global feature extractor and fixed sparsity ratios.
Experimental setup
Datasets: Vimeo90K X4K1000FPS (X-Train) X-Test Xiph SNU-FILM
Benchmarks: X-Test-L Xiph-L SNU-FILM-L SNU-FILM easy SNU-FILM medium
Metrics: PSNR SSIM
Baselines: XVFI FILM BiFormer RIFE EMA-VFI-small VFIFormer
Builds on: RIFE EMA-VFI GMFlow RAFT
Closest papers by meaning
- VTinker: Guided Flow Upsampling and Texture Mapping for High-Resolution Video Frame Interpolation (AAAI 2026 · similarity 0.88)
- ST-MFNet: A Spatio-Temporal Multi-Flow Network for Frame Interpolation (CVPR 2022 · similarity 0.88)
- Perception-Oriented Video Frame Interpolation via Asymmetric Blending (CVPR 2024 · similarity 0.88)
- BiM-VFI: Bidirectional Motion Field-Guided Frame Interpolation for Video with Non-uniform Motions (CVPR 2025 · similarity 0.88)
- EDEN: Enhanced Diffusion for High-quality Large-motion Video Frame Interpolation (CVPR 2025 · similarity 0.87)
- Disentangled Motion Modeling for Video Frame Interpolation (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.