Research map / Video / Video segmentation and tracking
SportsSloMo: A New Benchmark and Baselines for Human-centric Video Frame Interpolation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces SportsSloMo, a 130K-clip high-resolution human-centric slow-motion sports benchmark for video frame interpolation, and adds model-agnostic human segmentation and keypoint losses that improve seven VFI models.
Problem
Video frame interpolation methods are benchmarked mainly on non-human-centric datasets, and they degrade on human-centric sports videos with deformable bodies and frequent occlusion.
Approach
Provide a large, high-resolution, human-centric slow-motion sports VFI benchmark and plug model-agnostic auxiliary losses into existing VFI models: panoptic segmentation consistency (Mask2Former) and keypoint heatmap consistency (ViTPose) computed on synthesized versus ground-truth intermediate frames.
Key results
- EMA-VFI + human-aware loss reaches 30.75 PSNR / 0.952 SSIM / 4.19 IE on SportsSloMo vs 30.70 / 0.949 / 4.28 without it.
- SuperSloMo improves from 29.77 to 30.24 PSNR with the human-aware losses, and EBME from 30.15 to 30.48.
- Top methods score about 30.15-30.70 PSNR on SportsSloMo versus 36.19-36.64 on Vimeo90K, indicating the benchmark is harder.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add human-aware segmentation and keypoint losses | EMA-VFI · SportsSloMo test | PSNR: 30.70 -> 30.75 | Helped |
| Add human-aware segmentation and keypoint losses | EMA-VFI · SportsSloMo test | SSIM: 0.949 -> 0.952 | Helped |
| Add human-aware segmentation and keypoint losses | EMA-VFI · SportsSloMo test | IE: 4.28 -> 4.19 | Helped |
| Add segmentation loss only (ablation) | SuperSloMo · SportsSloMo test | PSNR: 29.77 -> 30.19 | Helped |
| Add keypoint loss only (ablation) | SuperSloMo · SportsSloMo test | PSNR: 29.77 -> 30.22 | Helped |
| Add segmentation and keypoint losses (ablation) | SuperSloMo · SportsSloMo test | PSNR: 29.77 -> 30.24 | Helped |
| Add segmentation loss only (ablation) | EBME · SportsSloMo test | PSNR: 30.15 -> 30.26 | Helped |
| Add keypoint loss only (ablation) | EBME · SportsSloMo test | PSNR: 30.15 -> 30.34 | Helped |
| Add segmentation and keypoint losses (ablation) | EBME · SportsSloMo test | PSNR: 30.15 -> 30.48 | Helped |
| Add GMFlow optical-flow distillation | SuperSloMo and EBME · SportsSloMo test | PSNR: 29.77 vs. 30.18; 30.15 vs. 29.85 | Mixed |
Surprising findings
- Optical-flow distillation from GMFlow helped SuperSloMo (29.77 to 30.18 PSNR) but hurt EBME (30.15 to 29.85 PSNR).
- Fine-tuning GMFlow on a synthetic human flow dataset caused poor flow estimation due to domain gap to real-world test images.
Limitations
- Only 2D human cues are used; 3D human reconstruction is not explored.
- Fast-moving balls and sports equipment are not modeled by the losses.
- Optical-flow distillation gives mixed results across models.
Experimental setup
Datasets: SportsSloMo Vimeo90K SNU-FILM COCO
Benchmarks: SportsSloMo
Metrics: PSNR SSIM IE
Baselines: SuperSloMo AdaCoF M2MVFI RIFE EBME UPR-Net EMA-VFI
Builds on: Charbonnier loss census loss Mask2Former ViTPose perceptual loss
Closest papers by meaning
- LAVIB: A Large-scale Video Interpolation Benchmark (NeurIPS 2024 · similarity 0.86)
- Repurposing Pre-trained Video Diffusion Models for Event-based Video Interpolation (CVPR 2025 · similarity 0.83)
- EDEN: Enhanced Diffusion for High-quality Large-motion Video Frame Interpolation (CVPR 2025 · similarity 0.83)
- Framer: Interactive Frame Interpolation (ICLR 2025 · similarity 0.83)
- Generative Inbetweening: Adapting Image-to-Video Models for Keyframe Interpolation (ICLR 2025 · similarity 0.82)
- Disentangled Motion Modeling for Video Frame Interpolation (AAAI 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.