Research map / 3D vision / Gaussian splatting
RoGSplat: Learning Robust Generalizable Human Gaussian Splatting from Sparse Multi-View Images
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Generalizable feed-forward human Gaussian Splatting that lifts SMPL vertices into dense image-aligned prior points (via pixel- and voxel-level features) and then regresses coarse-to-fine pixel-wise Gaussians from sparse multi-view images.
Problem
Novel view synthesis of unseen humans from sparse multi-view images with little overlap, where fitted SMPL templates are unreliable and depth-based Gaussian methods fail, without per-subject optimization.
Approach
Lift SMPL vertices to dense image-aligned prior points using an SPD network conditioned on both pixel-aligned image features and voxel features from a refined SMPL depth, then render depth from coarse Gaussians to unproject pixel-wise points for fine Gaussian regression.
Key results
- In-domain on THuman2.0 with 4 views, RoGSplat reaches 28.94 PSNR / 0.9615 SSIM / 0.0433 LPIPS, above GPS-Gaussian (27.79) and GHG (24.98).
- Cross-domain (trained on THuman2.0) on ZJU-MoCap reaches 30.80 PSNR vs. 29.57 for TransHuman, and 25.99 PSNR on real-world data vs. 24.60 for TransHuman.
- Ablation: removing both pixel- and voxel-level features drops PSNR from 28.94 to 27.41; removing coarse-to-fine regression drops it to 28.00.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add pixel-level and voxel-level features (ablation) | RoGSplat · THuman2.0 | PSNR: 27.41 -> 28.94 | Helped |
| Add pixel-level features (ablation) | RoGSplat · THuman2.0 | PSNR: 28.45 -> 28.94 | Helped |
| Add voxel-level features (ablation) | RoGSplat · THuman2.0 | PSNR: 28.28 -> 28.94 | Helped |
| Add coarse-to-fine Gaussian regression (ablation) | RoGSplat · THuman2.0 | PSNR: 28.00 -> 28.94 | Helped |
| Add offset estimator (ablation) | RoGSplat · THuman2.0 | PSNR: 28.56 -> 28.94 | Helped |
| Use RoGSplat instead of TransHuman | Generalizable human rendering · ZJU-MoCap | SSIM: 0.9656 -> 0.9623 | Hurt |
| Use RoGSplat instead of TransHuman | Generalizable human rendering · RenderPeople (cross-domain) | LPIPS: 0.0640 -> 0.0661 | Hurt |
Surprising findings
- Using GT SMPL gives other methods a large boost, while RoGSplat produces similar quality from inaccurate fitted SMPL, indicating robustness to template misalignment.
Limitations
- Fails to handle loose clothing, with too few prior points obtained.
- Struggles to recover high-frequency details for face and hands.
- Cannot animate or edit reconstructed models and accepts only fixed-view images.
Experimental setup
Datasets: THuman2.0 RenderPeople ZJU-MoCap real-world dataset from GPS-Gaussian
Benchmarks: THuman2.0 RenderPeople ZJU-MoCap real-world dataset (GPS-Gaussian)
Metrics: PSNR SSIM LPIPS
Baselines: SHERF NHP GP-NeRF TransHuman GPS-Gaussian GHG 3DGS
Builds on: 3D Gaussian Splatting Snowflake Point Deconvolution GPS-Gaussian TransHuman
Closest papers by meaning
- HumanSplat: Generalizable Single-Image Human Gaussian Splatting with Structure Priors (NeurIPS 2024 · similarity 0.90)
- Generalizable Human Gaussians from Single-View Image (ICLR 2025 · similarity 0.89)
- HiSplat: Hierarchical 3D Gaussian Splatting for Generalizable Sparse-View Reconstruction (ICLR 2025 · similarity 0.89)
- Guess The Unseen: Dynamic 3D Scene Reconstruction from Partial 2D Glimpses (CVPR 2024 · similarity 0.89)
- TranSplat: Generalizable 3D Gaussian Splatting from Sparse Multi-View Images with Transformers (AAAI 2025 · similarity 0.89)
- PF3plat: Pose-Free Feed-Forward 3D Gaussian Splatting for Novel View Synthesis (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.