Research map / 3D vision / Gaussian splatting
HiMoR: Monocular Deformable Gaussian Reconstruction with Hierarchical Motion Representation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HiMoR, a tree-structured hierarchical SE(3) motion representation with shared motion bases for deforming 3D Gaussians, plus CLIP-based perceptual evaluation, for monocular dynamic scene reconstruction.
Problem
Monocular dynamic 3D reconstruction with Gaussian splatting is ill-posed; existing deformation models either miss fine motion details or overfit, and pixel metrics like PSNR can misrepresent quality under misalignment.
Approach
A tree of motion nodes where each node stores motion relative to its parent, with children sharing a few SE(3) motion bases; coarse shallow nodes give smooth motion and deep nodes add detail, with depth-dependent rigidity loss and curve-distance-based node densification. Gaussians deform by skinning weights over KNN leaf nodes.
Key results
- On the iPhone dataset, HiMoR reaches mean CLIP-I 0.8907 and CLIP-T 0.9744 versus SoM's 0.8603 and 0.9712.
- On the Nvidia dataset, HiMoR achieves PSNR 23.400 and SSIM 0.7621, slightly above SoM (23.290, 0.7565).
- Ablation on the Spin scene shows motion bases, hierarchical structure, and rigidity loss each improve CLIP-I from 0.8677 baseline to 0.8853 in the full model.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add shared motion bases (ablation) | HiMoR baseline · iPhone dataset, Spin | CLIP-I: 0.8677 -> 0.8736 | Helped |
| add shared motion bases (ablation) | HiMoR baseline · iPhone dataset, Spin | CLIP-T: 0.9554 -> 0.9634 | Helped |
| add hierarchical structure (ablation) | HiMoR with motion bases · iPhone dataset, Spin | CLIP-I: 0.8736 -> 0.8753 | Helped |
| add hierarchical structure (ablation) | HiMoR with motion bases · iPhone dataset, Spin | CLIP-T: 0.9634 -> 0.9649 | Helped |
| add rigidity loss (ablation) | HiMoR with hierarchical structure · iPhone dataset, Spin | CLIP-I: 0.8753 -> 0.8851 | Helped |
| add node densification (ablation) | HiMoR with rigidity loss · iPhone dataset, Spin | CLIP-I: 0.8851 -> 0.8853 | Helped |
| add node densification (ablation) | HiMoR with rigidity loss · iPhone dataset, Spin | CLIP-T: 0.9653 -> 0.9658 | Helped |
Surprising findings
- Pixel-level PSNR can favor blurry or broken reconstructions over geometrically faithful ones, so the authors argue for perceptual CLIP-based metrics.
Limitations
- Struggles to represent parts absent from the canonical frame, such as newly added objects or newly exposed regions.
- Relies on pretrained segmentation, depth, and 2D tracking models for initialization and supervision.
- LPIPS and PSNR remain sensitive to misalignment, so the proposed CLIP metrics are an alternative rather than a fix.
Experimental setup
Datasets: iPhone dataset Nvidia dataset
Benchmarks: iPhone dataset Nvidia dataset
Metrics: CLIP-I CLIP-T LPIPS PCK-T PSNR SSIM
Baselines: T-NeRF HyperNeRF Deformable 3DGS Marbles SoM
Builds on: 3D Gaussian Splatting Shape of Motion MoSca Deformable 3D Gaussians
Closest papers by meaning
- World from Motion: Generative Dynamic Gaussian Reconstruction from Monocular Video (NeurIPS 2026 · similarity 0.88)
- 4D3R: Motion-Aware Neural Reconstruction and Rendering of Dynamic Scenes from Monocular Videos (NeurIPS 2025 · similarity 0.88)
- Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstruction (CVPR 2024 · similarity 0.87)
- Motion Decoupled 3D Gaussian Splatting for Dynamic Object Representation (AAAI 2025 · similarity 0.87)
- Neural Parametric Gaussians for Monocular Non-Rigid Object Reconstruction (CVPR 2024 · similarity 0.86)
- HiSplat: Hierarchical 3D Gaussian Splatting for Generalizable Sparse-View Reconstruction (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.