Research map / 3D vision / Gaussian splatting
GauSTAR: Gaussian Surface Tracking and Reconstruction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
GauSTAR binds 3D Gaussians to mesh faces to track and reconstruct dynamic surfaces from multi-view RGB-D video, unbinding Gaussians and re-meshing where topology changes.
Problem
Tracking and reconstructing dynamic surfaces with 3D Gaussians is hard when topology changes (surfaces appearing, disappearing, or splitting) while keeping tracking of persistent surfaces.
Approach
Gaussian Surfaces bind N Gaussians per mesh face so the mesh tracks surfaces; in regions with detected topology change, Gaussians are unbound from faces via a learned unbinding weight, and the mesh is re-meshed via TSDF fusion, combined with surface-based scene flow warping for initialization.
Key results
- GauSTAR reaches 31.87 PSNR, 0.102 LPIPS and 0.237 cm Chamfer Distance, outperforming HumanRF (30.59 PSNR, 0.284 CD) and 2DGS.
- Tracking 3D ATE of 0.452 cm vs 3.15 cm for Dynamic 3D Gaussians and 8.98 cm for PhysAvatar-SMPLX.
- Ablations show removing unbinding, re-meshing, or scene flow raises 3D ATE to 2.85, 2.08, and 6.56 cm respectively.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GauSTAR instead of HumanRF | Dynamic surface reconstruction · Captured sequences | PSNR: 30.59 -> 31.87 | Helped |
| Use GauSTAR instead of HumanRF | Dynamic surface reconstruction · Captured sequences | SSIM: 0.947 -> 0.952 | Helped |
| Use GauSTAR instead of HumanRF | Dynamic surface reconstruction · Captured sequences | LPIPS: 0.128 -> 0.102 | Helped |
| Use GauSTAR instead of HumanRF | Dynamic surface reconstruction · Captured sequences | CD (cm): 0.284 -> 0.237 | Helped |
| Use GauSTAR instead of 2D Gaussian Splatting | Dynamic surface reconstruction · Captured sequences | F-Score: 0.946 -> 0.980 | Helped |
| Use GauSTAR instead of Dynamic 3D Gaussians | Dynamic surface tracking · Captured sequences | 3D ATE: 3.15 -> 0.452 | Helped |
| Use GauSTAR instead of Dynamic 3D Gaussians | Dynamic surface tracking · Captured sequences | 2D ATE: 13.84 -> 2.03 | Helped |
| Add Gaussian unbinding (ablation) | GauSTAR · Captured sequences | CD (cm): 0.411 -> 0.237 | Helped |
| Add surface re-meshing (ablation) | GauSTAR · Captured sequences | CD (cm): 0.418 -> 0.237 | Helped |
| Add scene flow warping (ablation) | GauSTAR · Captured sequences | 3D ATE: 6.56 -> 0.45 | Helped |
Surprising findings
- 2DGS applied per frame without temporal information converges to artifacts in several frames.
Limitations
- Struggles with complex or sudden topology changes such as a new person entering the scene.
- Requires multi-view video data, limiting applicability in general public settings.
- Transparent and specular surfaces are challenging for Gaussian-based surface reconstruction.
Experimental setup
Datasets: in-house multi-view RGB-D capture (52 RGB + 52 IR cameras) AprilTag tracking sequences
Benchmarks: in-house multi-view capture sequences AprilTag tracking sequences
Metrics: PSNR SSIM LPIPS Chamfer Distance F-Score 3D ATE 2D ATE
Baselines: HumanRF Dynamic 3D Gaussians PhysAvatar-general PhysAvatar-SMPLX 2D Gaussian Splatting
Builds on: 3D Gaussian Splatting 2D Gaussian Splatting PhysAvatar TSDF fusion
Closest papers by meaning
- SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Rendering (CVPR 2024 · similarity 0.87)
- GaussianPainter: Painting Point Cloud into 3D Gaussians with Normal Guidance (AAAI 2025 · similarity 0.87)
- MAtCha Gaussians: Atlas of Charts for High-Quality Geometry and Photorealism From Sparse Views (CVPR 2025 · similarity 0.86)
- VA-GS: Enhancing the Geometric Representation of Gaussian Splatting via View Alignment (NeurIPS 2025 · similarity 0.86)
- Textured Gaussians for Enhanced 3D Scene Appearance Modeling (CVPR 2025 · similarity 0.86)
- Dynamic Gaussians Mesh: Consistent Mesh Reconstruction from Dynamic Scenes (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.