Research map / 3D vision / Depth and stereo
GFlow: Recovering 4D World from Monocular Video
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
GFlow lifts a single unposed monocular video into a dynamic 3D Gaussian scene plus per-frame camera poses by alternating camera and Gaussian optimization guided by depth and optical flow priors.
Problem
Recovering a dynamic 3D world and camera poses from a single monocular video without known camera parameters, multi-view input, or static-scene assumptions.
Approach
Uses only 2D priors (depth, optical flow, intrinsics) to model the video as a flow of 3D Gaussians; separates still and moving Gaussians via epipolar error, alternates per-frame camera-extrinsic optimization (static parts only) with Gaussian optimization, and adds prior-driven initialization and pixel-wise densification for new content.
Key results
- On 30 DAVIS 2017 videos, GFlow reaches 29.74 PSNR, 0.9205 SSIM, 0.1237 LPIPS, versus best baseline CoDeF at 24.89 PSNR, 0.7703 SSIM, 0.2932 LPIPS.
- Removing the isotropic loss drops DAVIS PSNR from 29.74 to 26.32.
- On MPI Sintel camera pose, GFlow achieves ATE 0.124 and RPE translation 0.039 (best RPE_t), though RoDynRF has better ATE (0.089).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use GFlow instead of CoDeF | 4D video reconstruction · DAVIS 2017 evaluation videos | PSNR: 24.89 -> 29.74 | Helped |
| use GFlow instead of CoDeF | 4D video reconstruction · DAVIS 2017 evaluation videos | SSIM: 0.7703 -> 0.9205 | Helped |
| use GFlow instead of CoDeF | 4D video reconstruction · DAVIS 2017 evaluation videos | LPIPS: 0.2932 -> 0.1237 | Helped |
| use GFlow instead of RoDynRF | camera pose estimation · MPI Sintel | ATE: 0.089 -> 0.124 | Hurt |
| use GFlow instead of RoDynRF | camera pose estimation · MPI Sintel | RPE translation: 0.073 -> 0.039 | Helped |
| use GFlow instead of RoDynRF | camera pose estimation · MPI Sintel | RPE rotation: 1.313 -> 0.599 | Helped |
| optimize camera poses rather than use MASt3R poses directly (ablation) | GFlow · DAVIS 2017 evaluation videos | PSNR: 29.21 -> 29.74 | Helped |
| add depth loss (ablation) | GFlow · DAVIS 2017 evaluation videos | PSNR: 29.30 -> 29.74 | Helped |
| add isotropic loss (ablation) | GFlow · DAVIS 2017 evaluation videos | PSNR: 26.32 -> 29.74 | Helped |
Surprising findings
- Directly using MASt3R camera poses (GFlow*) yields only slightly lower PSNR (29.21 vs 29.74), but the authors argue camera poses and motion are still wrong in dynamic scenes with flowing water.
- GFlow reaches the best DAVIS reconstruction metrics without any learned training, relying on per-video optimization.
Limitations
- Camera pose estimation depends on static regions; MASt3R fails when most of the frame is moving, and the method still needs pose initialization from MASt3R.
- Hyperparameters may need per-video tuning for best results.
- Many components rely on off-the-shelf priors (UniMatch, MASt3R), so errors in these priors propagate.
Experimental setup
Datasets: DAVIS 2017 MPI Sintel
Benchmarks: DAVIS MPI Sintel
Metrics: PSNR SSIM LPIPS ATE RPE translation RPE rotation
Baselines: Deformable Sprites RoDynRF CoDeF 4DGS R-CVD DROID-SLAM COLMAP NeRF-- BARF
Builds on: 3D Gaussian Splatting UniMatch MASt3R CoDeF
Closest papers by meaning
- World from Motion: Generative Dynamic Gaussian Reconstruction from Monocular Video (NeurIPS 2026 · similarity 0.89)
- Instant4D: 4D Gaussian Splatting in Minutes (NeurIPS 2025 · similarity 0.87)
- CAT4D: Create Anything in 4D with Multi-View Video Diffusion Models (CVPR 2025 · similarity 0.87)
- 4D3R: Motion-Aware Neural Reconstruction and Rendering of Dynamic Scenes from Monocular Videos (NeurIPS 2025 · similarity 0.86)
- Motion4D: Learning 3D-Consistent Motion and Semantics for 4D Scene Understanding (NeurIPS 2025 · similarity 0.86)
- 4DGT: Learning a 4D Gaussian Transformer Using Real-World Monocular Videos (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.