Research map / 3D vision / Gaussian splatting
Orientation-anchored Hyper-Gaussian for 4D Reconstruction from Casual Videos
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes OriGS, which propagates principal forward orientations over a monocular video into a Global Orientation Field and uses an orientation-aware Hyper-Gaussian with conditioned slicing to model region-specific deformations for 4D reconstruction.
Problem
4D reconstruction from casually captured monocular videos, where low-rank motion anchors (graph nodes, spline control points, motion bases) fail to capture complex, region-specific local deformations.
Approach
Uses scene orientation, estimated by PCA on trajectories and propagated via localized Procrustes alignment, as a dynamic anchor; embeds time, space, geometry and orientation into a hyper-Gaussian probabilistic state whose conditional mean (Gaussian slicing) gives per-Gaussian deformation given target time and orientation.
Key results
- On DyCheck average over seven scenes, OriGS with ground-truth poses reaches PSNR 19.69, SSIM 0.716, LPIPS 0.256, outperforming MoSca (18.84, 0.676, 0.289).
- Without ground-truth poses, OriGS averages PSNR 19.30, SSIM 0.685, LPIPS 0.270, also above MoSca's 18.84 / 0.676 / 0.289.
- On point tracking correspondence (PCK-T @0.05) on DyCheck, OriGS scores 0.851 versus MoSca 0.824 and CoTracker 0.803.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use OriGS instead of MoSca | DyCheck novel view synthesis, with camera pose · DyCheck average | PSNR: 18.84 -> 19.69 | Helped |
| use OriGS instead of MoSca | DyCheck novel view synthesis, with camera pose · DyCheck average | SSIM: 0.676 -> 0.716 | Helped |
| use OriGS instead of MoSca | DyCheck novel view synthesis, with camera pose · DyCheck average | LPIPS: 0.289 -> 0.256 | Helped |
| use OriGS instead of MoSca | DyCheck point tracking · DyCheck | PCK-T @0.05%: 0.824 -> 0.851 | Helped |
| provide ground-truth camera pose | OriGS · DyCheck average | PSNR: 19.30 -> 19.69 | Mixed |
| provide ground-truth camera pose | OriGS · DyCheck average | SSIM: 0.685 -> 0.716 | Helped |
| provide ground-truth camera pose | OriGS · DyCheck average | LPIPS: 0.270 -> 0.256 | Helped |
| replace the shared deformation MLP with anchor-driven deformation guided by the Global Orientation Field (ablation) | 3DGS-MLP · Apple (DyCheck, with camera pose) | PSNR: 13.47 -> 16.28 | Helped |
| replace the shared deformation MLP with anchor-driven deformation guided by the Global Orientation Field (ablation) | 3DGS-MLP · Apple (DyCheck, with camera pose) | SSIM: 0.532 -> 0.694 | Helped |
| replace the shared deformation MLP with anchor-driven deformation guided by the Global Orientation Field (ablation) | 3DGS-MLP · Apple (DyCheck, with camera pose) | LPIPS: 0.586 -> 0.476 | Helped |
| add a time-conditioned Hyper-Gaussian state over time, space, and geometry (ablation) | Deform w/ GOF · Apple (DyCheck, with camera pose) | PSNR: 16.28 -> 18.71 | Helped |
| add a time-conditioned Hyper-Gaussian state over time, space, and geometry (ablation) | Deform w/ GOF · Apple (DyCheck, with camera pose) | SSIM: 0.694 -> 0.750 | Helped |
Limitations
- Gains are limited for scenes with simple near-rigid motion where orientation stays roughly constant, while the high-dimensional modeling still adds optimization cost.
- Relies on 2D point trajectories and depth priors from foundation models, whose reliability affects reconstruction quality.
- Ground-truth pose inaccuracies in DyCheck (e.g., Space Out scene) limit performance.
Experimental setup
Datasets: DAVIS OpenAI SORA videos YouTube-VOS DyCheck
Benchmarks: DyCheck DAVIS OpenAI SORA YouTube-VOS
Metrics: PSNR SSIM LPIPS PCK-T
Baselines: T-NeRF NSFF Nerfies HyperNeRF RoDynRF DynPoint PGDVS DyBluRF CTNeRF Dynamic 3DGS 4DGS D-NPC
Builds on: 3D Gaussian Splatting Embedded Deformation Graphs MoSca 4DGS 6DGS
Closest papers by meaning
- MoDec-GS: Global-to-Local Motion Decomposition and Temporal Interval Adjustment for Compact Dynamic 3D Gaussian Splatting (CVPR 2025 · similarity 0.91)
- 4D Gaussian Splatting in the Wild with Uncertainty-Aware Regularization (NeurIPS 2024 · similarity 0.91)
- HAIF-GS: Hierarchical and Induced Flow-Guided Gaussian Splatting for Dynamic Scene (NeurIPS 2025 · similarity 0.91)
- Motion Decoupled 3D Gaussian Splatting for Dynamic Object Representation (AAAI 2025 · similarity 0.90)
- Fully Explicit Dynamic Gaussian Splatting (NeurIPS 2024 · similarity 0.90)
- Learning Efficient Fuse-and-Refine for Feed-Forward 3D Gaussian Splatting (NeurIPS 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.