Research map / 3D vision / Gaussian splatting
GPS-Gaussian: Generalizable Pixel-wise 3D Gaussian Splatting for Real-time Human Novel View Synthesis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
GPS-Gaussian regresses pixel-wise Gaussian parameter maps on two source views, lifted to 3D with a jointly trained iterative stereo depth estimator, enabling feed-forward real-time 2K human novel view synthesis.
Problem
Human novel view synthesis under sparse camera views typically requires per-subject optimization (3D Gaussian Splatting, NeRF) which is too slow for interactive use; generalizable methods are either slow or lack quality.
Approach
Define Gaussian parameter maps on source image planes (position from an iterative stereo depth map, color from RGB, rotation/scale/opacity regressed by a U-Net-style decoder with a depth encoder) and train depth and Gaussian regression jointly end-to-end through differentiable splatting.
Key results
- On THuman2.0, GPS-Gaussian reaches PSNR 25.57 / SSIM 0.898 / LPIPS 0.112 versus ENeRF 24.10 / 0.869 / 0.126 and 3D-GS 24.18 / 0.821 / 0.144.
- Renders 2K-resolution novel views at about 25 FPS on an RTX 3090 with two 1024x1024 source images, versus ENeRF at 5 FPS and FloRen at 15 FPS.
- Joint training of depth and Gaussian regression improves synthetic PSNR from 23.97 to 25.05 and depth EPE from 1.587 to 1.494.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GPS-Gaussian instead of ENeRF | Human novel view synthesis · THuman2.0 | PSNR: 24.10 -> 25.57 | Helped |
| Use GPS-Gaussian instead of ENeRF | Human novel view synthesis · Twindom | PSNR: 23.64 -> 24.79 | Helped |
| Use GPS-Gaussian instead of ENeRF | Human novel view synthesis · Real-world Data | PSNR: 23.26 -> 24.64 | Helped |
| Use GPS-Gaussian instead of FloRen | Novel view rendering from two 1024 × 1024 source images · Novel view rendering | FPS: 15 -> 25 | Helped |
| Use GPS-Gaussian instead of ENeRF | Human novel view synthesis · 6-camera setup | PSNR: 21.78 -> 23.03 | Helped |
| Add joint training with differentiable Gaussian rendering (ablation) | GPS-Gaussian · Synthetic data | PSNR: 23.97 -> 25.05 | Helped |
| Add joint training with differentiable Gaussian rendering (ablation) | GPS-Gaussian · Synthetic data | EPE: 1.587 -> 1.494 | Helped |
| Add a depth encoder (ablation) | GPS-Gaussian · Synthetic data | PSNR: 23.84 -> 25.05 | Helped |
| Add a depth encoder (ablation) | GPS-Gaussian · Synthetic data | LPIPS: 0.204 -> 0.121 | Helped |
Surprising findings
- Per-subject optimized 3D-GS with 8 views performs worse than the feed-forward method and yields noisy renderings in the sparse-view setting.
Limitations
- Requires accurate foreground matting as preprocessing
- Needs ground-truth depth for supervision, making training data acquisition harder
- Designed for human performers and not straightforwardly generalizable to general scenes
Experimental setup
Datasets: THuman2.0 Twindom real-world captured data (4 characters, 8-camera setup)
Benchmarks: THuman2.0 Twindom real-world data
Metrics: PSNR SSIM LPIPS FPS EPE 1-pixel error ratio
Baselines: 3D-GS ENeRF FloRen IBRNet
Builds on: 3D Gaussian Splatting RAFT-Stereo ENeRF FloRen
Closest papers by meaning
- Generative Sparse-View Gaussian Splatting (CVPR 2025 · similarity 0.91)
- FewViewGS: Gaussian Splatting with Few View Matching and Multi-stage Training (NeurIPS 2024 · similarity 0.91)
- Spacetime Gaussian Feature Splatting for Real-Time Dynamic View Synthesis (CVPR 2024 · similarity 0.90)
- Structure Consistent Gaussian Splatting with Matching Prior for Few-shot Novel View Synthesis (NeurIPS 2024 · similarity 0.90)
- Speedy-Splat: Fast 3D Gaussian Splatting with Sparse Pixels and Sparse Primitives (CVPR 2025 · similarity 0.90)
- GaussianPro: 3D Gaussian Splatting with Progressive Propagation (ICML 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.