Research map / 3D vision / Gaussian splatting
Triplane Meets Gaussian Splatting: Fast and Generalizable Single-View 3D Reconstruction with Transformers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TGS, a feed-forward transformer framework that reconstructs a hybrid triplane-Gaussian representation from a single image for fast 3D Gaussian Splatting reconstruction and novel view synthesis.
Problem
Single-view 3D reconstruction is slow when based on SDS or per-shape diffusion optimization, and directly regressing 3D Gaussians from images is hard due to their non-structural, high-dimensional nature.
Approach
A point decoder predicts a coarse-to-dense explicit point cloud; a triplane decoder with geometry-aware encoding produces implicit features queried at each point to decode 3D Gaussian attributes, augmented with projection-aware image features, enabling feed-forward generalizable Gaussian Splatting.
Key results
- On GSO, geometry reconstruction CD 21.04 x10^-3 and IoU 0.401 versus One-2-3-45 at 75.56 and 0.338, in 0.14 s.
- Novel view synthesis PSNR 23.15, SSIM 0.87, LPIPS 0.13 versus Zero-1-2-3 17.57 and One-2-3-45 15.55, with 0.003 s rendering time.
- Ablation shows triplane-Gaussian (PSNR 23.15) beats native generalizable 3D Gaussians (18.56) and Triplane-NeRF (21.85).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Triplane-Gaussian instead of direct 3D Gaussian decoding (ablation) | TGS · GSO novel-view synthesis | PSNR: 18.56 -> 23.15 | Helped |
| use Triplane-Gaussian instead of Triplane-NeRF (ablation) | TGS · GSO novel-view synthesis | PSNR: 21.85 -> 23.15 | Helped |
| use projection-aware condition rather than PCL tokens for point upsampling (ablation) | TGS point cloud upsampling · GSO geometry reconstruction | CD ×10−3: 22.16 -> 21.04 | Helped |
| use global image features rather than projection-aware condition for point upsampling (ablation) | TGS point cloud upsampling · GSO geometry reconstruction | CD ×10−3: 21.04 -> 22.22 | Hurt |
| add projection-aware condition to the Gaussian decoder (ablation) | TGS · GSO novel-view synthesis | PSNR: 22.76 -> 23.15 | Helped |
| add geometry-aware encoding to the triplane decoder (ablation) | TGS · GSO novel-view synthesis | PSNR: 22.47 -> 23.15 | Helped |
| replace ground-truth 3D supervision with pseudo-point-cloud supervision (ablation) | TGS · GSO novel-view synthesis | PSNR: 23.15 -> 22.65 | Hurt |
Surprising findings
- Directly regressing generalizable 3D Gaussians from tokens performs worst, indicating the representation itself is hard to learn without the triplane-point hybrid.
Limitations
- Rendering quality depends on the initial predicted point cloud; severe geometry deviation cannot be recovered.
- Non-probabilistic model yields blurry backsides of objects.
- Point decoder and triplane decoder trained separately, causing misalignment without geometry-aware encoding.
Experimental setup
Datasets: Objaverse-LVIS Google Scanned Objects (GSO)
Benchmarks: Google Scanned Objects (GSO)
Metrics: Chamfer Distance Volume IoU PSNR SSIM LPIPS runtime
Baselines: Point-E Shap-E One-2-3-45 Zero-1-2-3 PixelNeRF Triplane-NeRF 3DG
Builds on: 3D Gaussian Splatting LRM PC2 Snowflake point deconvolution DINOv2
Closest papers by meaning
- DirectTriGS: Triplane-based Gaussian Splatting Field Representation for 3D Generation (CVPR 2025 · similarity 0.92)
- SplatFormer: Point Transformer for Robust 3D Gaussian Splatting (ICLR 2025 · similarity 0.90)
- Splatter Image: Ultra-Fast Single-View 3D Reconstruction (CVPR 2024 · similarity 0.89)
- Swift4D: Adaptive divide-and-conquer Gaussian Splatting for compact and efficient reconstruction of dynamic scene (ICLR 2025 · similarity 0.89)
- pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction (CVPR 2024 · similarity 0.88)
- Gaussian-Flow: 4D Reconstruction with Dynamic 3D Gaussian Particle (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.