Research map / 3D vision / Gaussian splatting
MotionGS: Exploring Explicit Motion Guidance for Deformable 3D Gaussian Splatting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MotionGS, a deformable 3D Gaussian Splatting framework that decouples optical flow into camera and motion flow to supervise Gaussian deformation and alternately refines camera poses for monocular dynamic scene reconstruction.
Problem
Deformable 3D Gaussian Splatting for dynamic scenes lacks explicit motion constraints on object motion and relies on possibly inaccurate COLMAP camera poses, causing optimization difficulties and degraded rendering in monocular video.
Approach
Decouple 2D optical flow into camera flow (from camera ego-motion, computed from rendered depth and poses) and motion flow (object-only), then supervise a rendered Gaussian flow with the motion flow via an L1 loss; additionally refine camera poses by learnable SE(3) residuals with photometric gradients, alternating with Gaussian optimization.
Key results
- On NeRF-DS, MotionGS achieves mean PSNR 24.54, SSIM 0.8656, LPIPS 0.1719, versus 23.61 PSNR for the Deformable-3DGS baseline.
- On HyperNeRF vrig subset, MotionGS reports mean PSNR 24.8 and SSIM 0.69, versus 22.5 PSNR for Deformable 3DGS.
- Ablation shows motion flow guidance alone raises PSNR from 23.61 to 24.12, and adding camera pose refinement reaches 24.54; direct optical flow supervision drops PSNR to 23.37.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add MotionGS motion guidance and pose refinement | Deformable 3DGS · NeRF-DS mean | PSNR: 23.61 -> 24.54 | Helped |
| add MotionGS motion guidance and pose refinement | Deformable 3DGS · NeRF-DS mean | SSIM: 0.8394 -> 0.8656 | Helped |
| add MotionGS motion guidance and pose refinement | Deformable 3DGS · HyperNeRF vrig mean | PSNR: 22.5 -> 24.8 | Helped |
| use optical flow guidance instead of decoupled motion flow guidance (ablation) | Deformable 3DGS · NeRF-DS | PSNR: 23.61 -> 23.37 | Hurt |
| add decoupled motion flow guidance (ablation) | Deformable 3DGS · NeRF-DS | PSNR: 23.61 -> 24.12 | Helped |
| add camera pose refinement to motion flow guidance (ablation) | Motion flow guidance · NeRF-DS | PSNR: 24.12 -> 24.54 | Helped |
| remove the motion mask (ablation) | MotionGS · NeRF-DS | PSNR: 23.61 -> 23.13 | Hurt |
| use MiDaS depth instead of rendered depth (ablation) | MotionGS · NeRF-DS | PSNR: 23.61 -> 23.58 | Hurt |
| replace GMFlow with FlowFormer (ablation) | MotionGS · NeRF-DS | PSNR: 23.61 -> 23.97 | Mixed |
| replace GMFlow with MDFlow (ablation) | MotionGS · NeRF-DS | PSNR: 23.61 -> 23.25 | Hurt |
| use self-supervised flow supervision loss (ablation) | MotionGS · NeRF-DS | PSNR: 23.61 -> 23.76 | Helped |
| lower flow loss weight to λ=0.2 (ablation) | MotionGS · NeRF-DS | PSNR: 23.61 -> 23.46 | Hurt |
Surprising findings
- Directly supervising Gaussian deformation with full optical flow degrades performance below the baseline (23.37 vs 23.61 PSNR), while decoupled motion flow improves it.
- Using monocular depth from MiDaS for camera flow yields no gain over baseline, whereas rendered depth from 3DGS works better.
Limitations
- Fails on DyNeRF dataset with fixed sparse viewpoints, where inaccurate rendered depth corrupts camera flow and causes floating artifacts.
- Training time and GPU memory increase relative to the baseline; still depends on COLMAP initialization of camera poses.
- Relies on off-the-shelf optical flow and dynamic masks from segmentation networks.
Experimental setup
Datasets: NeRF-DS HyperNeRF
Benchmarks: NeRF-DS HyperNeRF vrig
Metrics: PSNR SSIM LPIPS FPS training time GPU memory
Baselines: 3D-GS TiNeuVox HyperNeRF NeRF-DS Deformable-3DGS D-NeRF
Builds on: 3D Gaussian Splatting Deformable 3D Gaussians GMFlow Bundle Adjustment
Closest papers by meaning
- Motion Decoupled 3D Gaussian Splatting for Dynamic Object Representation (AAAI 2025 · similarity 0.93)
- H3D-DGS: Exploring Heterogeneous 3D Motion Representation for Deformable 3D Gaussian Splatting (NeurIPS 2025 · similarity 0.93)
- Gaussian Splatting Lucas-Kanade (ICLR 2025 · similarity 0.92)
- DGS-LRM: Real-Time Deformable 3D Gaussian Reconstruction From Monocular Videos (NeurIPS 2025 · similarity 0.92)
- Fully Explicit Dynamic Gaussian Splatting (NeurIPS 2024 · similarity 0.91)
- Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstruction (CVPR 2024 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.