Research map / Video / Video segmentation and tracking
BF-STVSR: B-Splines and Fourier---Best Friends for High Fidelity Spatial-Temporal Video Super-Resolution
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BF-STVSR, a continuous spatial-temporal video super-resolution framework using a B-spline Mapper for temporal motion and a Fourier Mapper for spatial frequencies, removing the need for a pre-trained optical flow network.
Problem
Continuous spatial-temporal video super-resolution (C-STVSR) with INR-based models struggles to capture complex video motion and high-frequency detail; naive position encoding degrades performance, especially when combined with pre-trained optical flow networks.
Approach
Replace coordinate concatenation and RAFT optical-flow guidance with axis-specific encoding: B-spline basis functions predict smooth temporal motion vectors directly from encoded video features, and Fourier bases (amplitude and frequency estimated from latent features) capture dominant spatial frequencies.
Key results
- On GoPro-Average (x4 spatial, x8 temporal), BF-STVSR reaches 30.22 dB / 0.8802 SSIM, surpassing MoTIF's 30.04 / 0.8773.
- On Adobe240-Average, BF-STVSR achieves 30.12 dB / 0.8808, versus MoTIF's 29.82 / 0.8750.
- Table 3 video quality: BF-STVSR gets VMAF 61.06 on GoPro and 70.82 on Adobe, above MoTIF's 59.82 and 68.40.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add B-spline Mapper and Fourier Mapper without a pre-trained optical-flow network | MoTIF · GoPro-Average and Adobe-Average | PSNR (dB): 30.04 -> 30.22; 29.82 -> 30.12 | Helped |
| Use a pre-trained optical-flow network with both proposed mappers (ablation) | BF-STVSR · GoPro-Average and Adobe-Average | PSNR (dB): 30.03 -> 29.94; 29.81 -> 29.73 | Hurt |
| Use B-spline Mapper without Fourier Mapper (ablation) | BF-STVSR · GoPro-Average and Adobe-Average | PSNR (dB): 30.22 -> 30.12; 30.12 -> 30.02 | Hurt |
| Use Fourier Mapper without B-spline Mapper (ablation) | BF-STVSR · GoPro-Average and Adobe-Average | PSNR (dB): 30.22 -> 30.16; 30.12 -> 30.11 | Hurt |
| Add optical-flow supervision (ablation) | BF-STVSR · GoPro-Average and Adobe-Average | PSNR (dB): 30.22 -> 30.20; 30.12 -> 30.14 | No clear effect |
Surprising findings
- Adding position encoding and pre-trained RAFT optical flow to MoTIF-style models degrades performance, contrary to common expectations.
- The method performs comparably without the RAFT optical flow supervision loss.
Limitations
- Existing C-STVSR models, including this one, still struggle with large motion.
- Training of C-STVSR models is time-consuming and computationally expensive.
Experimental setup
Datasets: Adobe240
Benchmarks: Vid4 GoPro Adobe240
Metrics: PSNR SSIM VFIPS FloLPIPS tOF VMAF
Baselines: VideoINR MoTIF TMNet ZoomingSloMo RIFE EMA-VFI LIIF LTE SuperSloMo QVI DAIN EDVR
Builds on: MoTIF VideoINR LTE BTC softmax splatting
Closest papers by meaning
- Time Without Time: Pseudo-Temporal Representation for Space-Time Super-Resolution (CVPR 2026 · similarity 0.91)
- Spatial-Temporal Space Hand-in-Hand: Spatial-Temporal Video Super-Resolution via Cycle-Projected Mutual Learning (CVPR 2022 · similarity 0.90)
- EvEnhancer: Empowering Effectiveness, Efficiency and Generalizability for Continuous Space-Time Video Super-Resolution with Events (CVPR 2025 · similarity 0.86)
- Sequence Matters: Harnessing Video Models in 3D Super-Resolution (AAAI 2025 · similarity 0.86)
- EvSTVSR: Event Guided Space-Time Video Super-Resolution (AAAI 2025 · similarity 0.86)
- Enhancing Video Super-Resolution via Implicit Resampling-based Alignment (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.