Research map / Video / Video segmentation and tracking
LC-Mamba: Local and Continuous Mamba with Shifted Windows for Frame Interpolation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LC-Mamba, a Mamba-based video frame interpolation model that uses shifted local windows, a Hilbert-curve selective scan (H-SS2D/H-SS3D), and interleaved spatiotemporal scanning, reaching 36.53 dB PSNR on Vimeo90K.
Problem
Video frame interpolation requires capturing fine-grained local motion and spatiotemporal continuity, but Mamba's 1D scanning of 2D frames causes historical decay and spatial discontinuity across scan boundaries.
Approach
Combine window-restricted selective scans with a Hilbert space-filling-curve ordering (so spatially adjacent pixels stay adjacent in the 1D sequence), add cyclic-shifted windows with masking to connect window boundaries, and interleave two frames' tokens for a 3D (voxel-level) spatiotemporal scan.
Key results
- Ours-P reaches 36.53 dB PSNR / 0.9816 SSIM on Vimeo90K, +0.03 dB over VFIFormer's 36.50 dB.
- Ours-E (6.7M params) gives 36.19 dB on Vimeo90K and 36.67 dB on Xiph-2K, with competitive results against models with more parameters.
- In the scan ablation, SW-H-SS3D gives the best average SNU-FILM score (32.89) and Xiph-4K score (34.26) among the scanning methods compared under the Ours-E configuration.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use the proposed LC-Mamba configuration | LC-Mamba · Vimeo90K | PSNR: 36.50 -> 36.53 | Helped |
| use the proposed LC-Mamba configuration | LC-Mamba · UCF101 | PSNR: 35.42 -> 35.42 | No clear effect |
| use the proposed LC-Mamba configuration | LC-Mamba · Xiph-2K | PSNR: 37.02 -> 36.99 | Hurt |
| use the proposed LC-Mamba configuration | LC-Mamba · Xiph-4K | PSNR: 34.54 -> 34.49 | Hurt |
| use the proposed LC-Mamba configuration | LC-Mamba · SNU-FILM | PSNR: 40.41 -> 40.16; 36.30 -> 36.17; 30.89 -> 30.72; 25.68 -> 25.48 | Hurt |
| replace alternative scans with window-based Hilbert scanning (ablation) | Ours-E · Xiph-4K | PSNR: 33.13 -> 34.26; 33.80 -> 34.26; 33.99 -> 34.26; 34.01 -> 34.26; 33.30 -> 34.26 | Helped |
| remove shifted windows at window size 8 (ablation) | Ours-B · Vimeo90K | PSNR: 36.43 -> 36.45 | Helped |
| increase window size to 16 and remove shifted windows (ablation) | Ours-B · Vimeo90K | PSNR: 36.43 -> 36.46 | Helped |
| increase channels from 16 to 32 while keeping N1 / N2 at 4 / 4 (ablation) | LC-Mamba blocks · Vimeo90K | PSNR: 36.19 -> 36.53 | Helped |
Surprising findings
- Omitting shifted windows improved performance at low resolution (Vimeo90K) in the window-setting ablation, and a window size of 16 without shifting performed comparably to shifted variants.
Limitations
- Paper acknowledges Mamba still has limitations in global modeling and long-range dependency.
- Ablations are run mainly on the Ours-E or Ours-B configurations rather than the full Ours-P model.
Experimental setup
Datasets: Vimeo90K UCF101 Middlebury SNU-FILM Xiph
Benchmarks: Vimeo90K UCF101 Middlebury SNU-FILM Xiph-2K Xiph-4K
Metrics: PSNR SSIM IE LPIPS FloLPIPS
Baselines: IFRNet M2M SoftSplat RIFE BMBC EMA-S VFIMamba-S VFIFormer-S ABME SGM-VFI-S-1/2 SepConv AdaCoF
Builds on: VMamba MixedSSM (VFIMamba) Swin Transformer Mamba RIFE
Closest papers by meaning
- Spatial-Mamba: Effective Visual State Space Models via Structure-Aware State Fusion (ICLR 2025 · similarity 0.89)
- VFIMamba: Video Frame Interpolation with State Space Models (NeurIPS 2024 · similarity 0.87)
- EventMamba: Enhancing Spatio-Temporal Locality with State Space Models for Event-Based Video Reconstruction (AAAI 2025 · similarity 0.87)
- Vision Mamba Mender (NeurIPS 2024 · similarity 0.86)
- DiMSUM: Diffusion Mamba - A Scalable and Unified Spatial-Frequency Method for Image Generation (NeurIPS 2024 · similarity 0.86)
- Framer: Interactive Frame Interpolation (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.