Research map / Video / Video segmentation and tracking
LAVIB: A Large-scale Video Interpolation Benchmark
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces LAVIB, a large-scale 4K 60fps video interpolation benchmark of 283K clips from YouTube, split by motion, sharpness, contrast, and luminance metrics with OOD challenges, and benchmarks RIFE, EMA-VFI, and FLAVR.
Problem
Existing video frame interpolation datasets are small, low-resolution, or lack variation in motion, luminance, sharpness, and contrast, limiting generalization evaluation for VFI.
Approach
Builds a large web-sourced 4K/60fps VFI dataset via an automated pipeline that scores clips and tubelets with per-video metrics (AFM, ALV, ARMS, ARL), balances splits with DUPLEX selection, and defines OOD challenges by splitting on these metrics.
Key results
- FLAVR pre-trained on LAVIB reaches 33.44 PSNR / 0.981 SSIM on the LAVIB test set, versus 32.10 / 0.963 when pre-trained on Vimeo-90K.
- Training FLAVR on LAVIB improves Vimeo-90K PSNR from 36.25 to 36.68 and Xiph4K from 33.28 to 34.51.
- RIFE performs much worse than FLAVR and EMA-VFI on LAVIB test (PSNR 27.88 vs 33.44 for FLAVR).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Pre-train FLAVR on LAVIB instead of Vimeo-90K | FLAVR · LAVIB test | PSNR: 32.10 -> 33.44 | Helped |
| Fine-tune FLAVR on Xiph + X4K1KFPS instead of pre-training on Vimeo-90K | FLAVR · LAVIB test | PSNR: 32.10 -> 31.78 | Hurt |
| Train FLAVR on LAVIB instead of Vimeo-90K | FLAVR · Vimeo-90K | PSNR: 36.25 -> 36.68 | Helped |
| Train FLAVR on LAVIB instead of Vimeo-90K | FLAVR · Xiph4K | PSNR: 33.28 -> 34.51 | Helped |
| Train FLAVR on LAVIB instead of Vimeo-90K | FLAVR · X4K1KFPS | PSNR: 31.25 -> 32.44 | Helped |
| Replace RIFE with FLAVR | VFI models evaluated on LAVIB test · LAVIB test | PSNR: 27.88 -> 33.44 | Helped |
| Reduce frame resolution from 256 × 256 to 224 × 224 (ablation) | FLAVR · LAVIB test | PSNR: 33.44 -> 33.28 | Hurt |
| Reduce frame resolution from 256 × 256 to 112 × 112 (ablation) | FLAVR · LAVIB test | PSNR: 33.44 -> 32.57 | Hurt |
| Use septuplets instead of triplets as input (ablation) | FLAVR · LAVIB test, ×2 interpolation | PSNR: 33.44 -> 33.62 | Helped |
| Train with 30% activity-query videos instead of no activity-query videos (ablation) | FLAVR · LAVIB test | PSNR: 28.57 -> 31.08 | Helped |
| Increase activity-query videos from 30% to 100% (ablation) | FLAVR · LAVIB test | PSNR: 31.08 -> 29.23 | Hurt |
| Train with 30% misc-query videos instead of no misc-query videos (ablation) | FLAVR · LAVIB test | PSNR: 29.31 -> 30.54 | Helped |
Surprising findings
- Training on a 700-video subset from a single location query performs close to larger training sets, suggesting low sensitivity to location diversity.
- Reducing input resolution from 256x256 to 112x112 causes only a small PSNR drop (about -0.87 for FLAVR), which the authors suggest makes LAVIB usable for low-compute evaluation.
Limitations
- Queries are drawn from a finite manually-defined vocabulary of search terms, limiting diversity of locations and activities.
- Metric thresholds for selection are manually defined and may constrain dataset size.
- Benchmark evaluates only three VFI models and a single FLAVR baseline setup, with no error bars across seeds.
Experimental setup
Datasets: LAVIB Vimeo-90K Xiph X4K1KFPS
Benchmarks: LAVIB Vimeo-90K Xiph4K X4K1KFPS
Metrics: PSNR SSIM LPIPS DISTS Watson-DFT VSFA VFIPS
Baselines: RIFE EMA-VFI FLAVR
Builds on: FLAVR DUPLEX MViTv2
Closest papers by meaning
- Q-Bench-Video: Benchmark the Video Quality Understanding of LMMs (CVPR 2025 · similarity 0.86)
- VFIMamba: Video Frame Interpolation with State Space Models (NeurIPS 2024 · similarity 0.86)
- Sparse Global Matching for Video Frame Interpolation with Large Motion (CVPR 2024 · similarity 0.86)
- SportsSloMo: A New Benchmark and Baselines for Human-centric Video Frame Interpolation (CVPR 2024 · similarity 0.86)
- InFlux: A Benchmark for Self-Calibration of Dynamic Intrinsics of Video Cameras (NeurIPS 2025 · similarity 0.86)
- EDEN: Enhanced Diffusion for High-quality Large-motion Video Frame Interpolation (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.