Research map / 3D vision / Depth and stereo
3D Multi-frame Fusion for Video Stabilization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RStab, a generalizable volume-rendering framework that fuses multi-frame color and feature information in 3D for full-frame, structure-preserving video stabilization.
Problem
Video stabilization must simultaneously generate full-frame (uncropped) outputs and preserve scene structure, which single-frame 2D or 3D methods and multi-frame 2D methods fail to do jointly; dynamic regions further corrupt multi-frame 3D fusion.
Approach
Stabilized Rendering fuses warped colors and features from neighboring frames via IBRNet-style volume rendering along epipolar rays; Adaptive Ray Range uses temporally weighted multi-frame depth priors to bound sampling, and Color Correction refines color projection with optical flow.
Key results
- On the NUS dataset, RStab achieves cropping ratio 1.00, distortion 0.91 and stability 0.94, outperforming all baselines on distortion and cropping.
- On the Selfie dataset, RStab reaches distortion 0.92 and stability 0.95, the best among compared stabilizers.
- Ablation on the Crowd scene: full model gets distortion 0.90 and PSNR 40.01, versus 23.56 PSNR without Stabilized Rendering and 35.81 without Color Correction.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add Stabilized Rendering (ablation) | RStab · NUS Crowd | Distortion Value: 0.87 -> 0.90 | Helped |
| Add Stabilized Rendering (ablation) | RStab · NUS Crowd | PSNR: 23.56 -> 40.01 | Helped |
| Add Adaptive Ray Range (ablation) | RStab · NUS Crowd | Distortion Value: 0.81 -> 0.90 | Helped |
| Add Adaptive Ray Range (ablation) | RStab · NUS Crowd | PSNR: 37.83 -> 40.01 | Helped |
| Add Color Correction (ablation) | RStab · NUS Crowd | Distortion Value: 0.86 -> 0.90 | Helped |
| Add Color Correction (ablation) | RStab · NUS Crowd | PSNR: 35.81 -> 40.01 | Helped |
Surprising findings
- Naive image blending of warped neighboring frames causes noticeable blur, and direct 3D projection aggregation without the proposed modules also blurs results.
Limitations
- Dynamic objects violate epipolar constraints, so the method relies on depth priors and optical flow that may be unreliable in dynamic regions.
- Per-frame sliding-window rendering with 13 neighboring frames and 3 samples per ray; efficiency not reported.
Experimental setup
Datasets: LLFF IBRNet-Collected NUS Selfie DeepStab
Benchmarks: NUS Selfie DeepStab
Metrics: Cropping Ratio Distortion Value Stability Score PSNR
Baselines: Grundmann et al. Bundle Yu and Ramamoorthi DIFRINT FuSta Zhao et al. Deep3D IBRNet ENeRF
Builds on: IBRNet ENeRF Deep3D NeRF SoftSplat
Closest papers by meaning
- SimVS: Simulating World Inconsistencies for Robust View Synthesis (CVPR 2025 · similarity 0.82)
- LinPrim: Linear Primitives for Differentiable Volumetric Rendering (NeurIPS 2025 · similarity 0.82)
- Inverse Rendering using Multi-Bounce Path Tracing and Reservoir Sampling (ICLR 2025 · similarity 0.82)
- Harnessing Meta-Learning for Improving Full-Frame Video Stabilization (CVPR 2024 · similarity 0.82)
- EditSplat: Multi-View Fusion and Attention-Guided Optimization for View-Consistent 3D Scene Editing with 3D Gaussian Splatting (CVPR 2025 · similarity 0.82)
- HiCoM: Hierarchical Coherent Motion for Dynamic Streamable Scenes with 3D Gaussian Splatting (NeurIPS 2024 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.