Research map / 3D vision / Gaussian splatting
Motion Decoupled 3D Gaussian Splatting for Dynamic Object Representation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes M5D-GS, a 3D Gaussian Splatting method that decouples object-level rigid motion (an MLP predicting a global transform) from per-Gaussian local deformation, with coarse-to-fine matching for large-motion initialization, and introduces a large-motion monocular dynamic dataset.
Problem
Dynamic object reconstruction from monocular video fails when objects undergo large motions, since joint canonical-space and deformation-field learning becomes ambiguous and the warm-up initialization yields empty or poor scenes.
Approach
Separate object-level motion (predicted rigid rotation and translation applied to canonical Gaussians) from per-Gaussian deformation, and use a coarse-to-fine matching warm-up that enlarges Gaussians and trains the motion block so that large initial displacements still yield overlap.
Key results
- On the large motion benchmark M5D-GS reaches PSNR 35.74 average over three scenes with 135.15 FPS, versus D3D-GS 27.12 PSNR at 267.39 FPS.
- Under added large motion, M5D-GS shows the smallest mean PSNR drop (3.06) versus D3D-GS (10.01) and 4D-GS (20.34).
- M5D-GS succeeds on all ten scenes where several baselines (e.g., SC-GS, 4D-GS, D3D-GS) fail on some.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use M5D-GS instead of TiNeuVox | Dynamic object representation · Jumping Jacks + Motion | PSNR: 29.22 -> 35.86 | Helped |
| Use M5D-GS instead of D3D-GS | Dynamic object representation · Hell Warrior + Motion | PSNR: 32.16 -> 37.37 | Helped |
| Use M5D-GS instead of K-Planes | Dynamic object representation · Bouncing Balls + Motion | PSNR: 32.60 -> 36.47 | Helped |
| Use M5D-GS instead of D3D-GS | Dynamic object representation · Elephant | PSNR: 29.00 -> 37.44 | Helped |
| Use M5D-GS instead of TiNeuVox | Dynamic object representation · Grey Cat | PSNR: 33.28 -> 39.69 | Helped |
| Use M5D-GS instead of TiNeuVox | Dynamic object representation · Robot | PSNR: 24.52 -> 29.78 | Helped |
| Use M5D-GS instead of D3D-GS | Dynamic object representation · Pokemon | PSNR: 29.84 -> 35.60 | Helped |
| Use M5D-GS instead of TiNeuVox | Dynamic object representation · Fish | PSNR: 27.33 -> 36.15 | Helped |
| Use M5D-GS instead of D3D-GS | Dynamic object representation · Robot Dog | PSNR: 19.25 -> 33.46 | Helped |
| Use M5D-GS instead of D3D-GS | Dynamic object representation · Pillow | PSNR: 29.89 -> 32.37 | Helped |
| Use M5D-GS instead of D3D-GS | 3D-GS rendering on three shared successful scenes · Three scenes where all 3D-GS methods succeeded | FPS: 267.39 -> 135.15 | Hurt |
| Add large motion to the original scenes | M5D-GS · Three upgraded scenes | Mean PSNR decrease: 3.06 | Mixed |
Surprising findings
- Many 3D-GS dynamic baselines fail completely on the added large-motion scenes, while NeRF-based TiNeuVox remains competitive on several of them.
Limitations
- Supports only a single object-level motion prediction, not separate motions for multiple parts or objects.
- Monocular setting is underconstrained, leading to overfitting risk.
- Evaluated mainly on synthetic and few real scenes (two real test cases).
Experimental setup
Datasets: D-NeRF HyperNeRF Yan, Li, and Lee 2023 dataset
Benchmarks: M5D-GS large motion benchmark D-NeRF
Metrics: PSNR SSIM LPIPS FPS
Baselines: D3D-GS SC-GS 4D-GS K-Planes TiNeuVox
Builds on: 3D Gaussian Splatting D3D-GS D-NeRF
Closest papers by meaning
- MotionGS: Exploring Explicit Motion Guidance for Deformable 3D Gaussian Splatting (NeurIPS 2024 · similarity 0.93)
- Fully Explicit Dynamic Gaussian Splatting (NeurIPS 2024 · similarity 0.93)
- H3D-DGS: Exploring Heterogeneous 3D Motion Representation for Deformable 3D Gaussian Splatting (NeurIPS 2025 · similarity 0.93)
- MoDec-GS: Global-to-Local Motion Decomposition and Temporal Interval Adjustment for Compact Dynamic 3D Gaussian Splatting (CVPR 2025 · similarity 0.93)
- SC-GS: Sparse-Controlled Gaussian Splatting for Editable Dynamic Scenes (CVPR 2024 · similarity 0.92)
- Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstruction (CVPR 2024 · similarity 0.92)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.