Research map / 3D vision / Gaussian splatting
HoliGS: Holistic Gaussian Splatting for Embodied View Synthesis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HoliGS, a deformable Gaussian splatting framework with hierarchical warping (global SE(3) poses, skeleton-driven DQB articulation, and an invertible RealNVP soft deformation) for embodied novel-view synthesis from minute-long monocular videos.
Problem
Embodied view synthesis of dynamic scenes with humans and animals from long monocular RGB videos, where prior NeRF and deformable Gaussian methods are costly, fragile under occlusion, and fail on extreme viewpoints.
Approach
Decomposes the scene into static background plus deformable foreground Gaussians, warped hierarchically by global rigid transforms, skeleton-driven dual-quaternion skinning, and an invertible RealNVP flow with cycle-consistency, initialized from a neural SDF rather than external point trackers.
Key results
- Mean across 11 sequences: PSNR 21.31, SSIM 0.747, LPIPS 0.263 for HoliGS, versus Total-Recon (w/ depth) at 18.11 PSNR and 0.724 SSIM.
- Depth mean Acc@0.1m 0.901 and RMS depth error 0.127 m, versus 0.731-ish for Total-Recon at 0.131 m error (Total-Recon mean error 0.131 m).
- Reports about 1.2 hours training versus about 12 hours for Total-Recon on the same A6000, roughly 10x faster.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add depth supervision (ablation) | HoliGS · Ablation study | Acc@0.1m: .847 -> .896 | Helped |
| Add normal supervision (ablation) | HoliGS · Ablation study | Acc@0.1m: .832 -> .896 | Helped |
| Add articulated deformable-object transform (ablation) | HoliGS · Ablation study | Acc@0.1m: .853 -> .896 | Helped |
| Add soft deformation (ablation) | HoliGS · Ablation study | Acc@0.1m: .870 -> .896 | Helped |
| Add root-body pose initialization (ablation) | HoliGS · Ablation study | Acc@0.1m: .862 -> .896 | Helped |
| Add initialization (ablation) | HoliGS · Total-Recon dataset mean | PSNR: 17.30 -> 21.31 | Helped |
| Remove object-specific rigid root motion (ablation) | HoliGS · Ablation study | convergence | Hurt |
| Train on the full HoliGS pipeline rather than Total-Recon | HoliGS vs. Total-Recon · Total-Recon dataset; RTX A6000 | training time: \textasciitilde1.2 hours vs. \textasciitilde 12 hours; $\approx\!10\times$ speed-up | Helped |
Surprising findings
- Removing root-body rigid transforms entirely prevents convergence, indicating the deformation field alone cannot model global object motion.
Limitations
- Generic pose estimation can mismatch anatomical accuracy of parametric models such as SMPL.
- Smooth deformation field assumption makes abrupt discontinuous motion hard to model.
- Dataset is small (11 sequences) and newly captured; code is only promised ('will be released').
Experimental setup
Datasets: newly captured stereo RGB dataset (11 sequences)
Benchmarks: newly captured stereo RGB dataset (11 sequences) Total-Recon dataset
Metrics: LPIPS PSNR SSIM Acc@0.1m RMS depth error
Baselines: HyperNeRF D^2NeRF Total-Recon Deformable-GS 4DGS GS-marble MoSca Shape-of-Motion
Builds on: Total-Recon Deformable 3D Gaussians RealNVP Omnimotion RAFT
Closest papers by meaning
- Human Gaussian Splatting: Real-time Rendering of Animatable Avatars (CVPR 2024 · similarity 0.89)
- Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstruction (CVPR 2024 · similarity 0.89)
- Orientation-anchored Hyper-Gaussian for 4D Reconstruction from Casual Videos (NeurIPS 2025 · similarity 0.89)
- SC-GS: Sparse-Controlled Gaussian Splatting for Editable Dynamic Scenes (CVPR 2024 · similarity 0.89)
- GauHuman: Articulated Gaussian Splatting from Monocular Human Videos (CVPR 2024 · similarity 0.88)
- COLMAP-Free 3D Gaussian Splatting (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.