Research map / 3D vision / Gaussian splatting
DirectTriGS: Triplane-based Gaussian Splatting Field Representation for 3D Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DirectTriGS, which represents 3D Gaussian Splatting objects as triplane fields, decodes them with a differentiable TriRenderer trained from 2D supervision, and generates them via a triplane VAE with two-stage latent diffusion plus optional SDS refinement.
Problem
Direct generation of 3D Gaussian Splatting content is hard because GS is a sparse, multi-channel point cloud with uneven density and non-Euclidean attributes, and prior generation methods are either slow SDS/2D-lifting or limited in direct GS generation.
Approach
Encode GS attributes and SDF geometry as image-like triplanes, use a fully differentiable TriRenderer (SDF via FlexiCubes, surface sampling of flat splats, GS rasterization) so that only multi-view 2D losses are needed, then run a triplane VAE and staged latent diffusion (geometry first, then GS appearance) for fast direct text-to-3D GS generation.
Key results
- User study overall score 3.423 for DirectTriGS vs 3.042 for Shap-E and 3.183 for Direct3D (raw LDM outputs).
- CLIP score (ViT-L-14) 0.2456 for DirectTriGS vs 0.2398 Shap-E and 0.2152 Direct3D.
- Generation takes 16.2 s per object on an RTX 3090 with 56.65M parameters and 2.94 GB GPU memory.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DirectTriGS instead of Direct3D | Text-to-3D generation · Generated 3D objects | Overall rating: 3.183 -> 3.423 | Helped |
| Use DirectTriGS instead of Shap-E | Text-to-3D generation · Text-to-3D | CLIP score (openai:ViT-L-14): 0.2398 -> 0.2456 | Helped |
| Apply SDS refinement to DirectTriGS outputs | DirectTriGS · Generated 3D objects | Overall (%): 42.5 -> 46.3 | Helped |
| Replace triplane-based representation with voxel occupancy for GS attribute generation (ablation) | DirectTriGS generation · Toy single-object overfitting experiment | GS attribute generation quality | Hurt |
| Remove the VAE and diffuse directly in triplane space (ablation) | DirectTriGS generation · Text-to-3D | Decoded geometry quality | Hurt |
Surprising findings
- Direct diffusion on triplanes without a VAE produced serious noise on decoded geometry.
- Attempting voxel-based GS attribute diffusion failed even when overfitting a single object, with the model learning color but not scaling, opacity, or orientation.
Limitations
- Evaluation relies on a small user study and CLIP score with no standard 3D generation metrics reported.
- Raw LDM outputs have limited texture quality, requiring optional SDS refinement which takes longer.
- Direct diffusion on GS attributes and triplanes without VAE failed, indicating the approach depends on the VAE compression.
Experimental setup
Datasets: Objaverse OmniObject3D
Benchmarks: Objaverse
Metrics: CLIP score user study ratings (geometry, texture, realism, consistency,
Baselines: Shap-E Direct3D GaussianCube GSGEN GaussianDreamer
Builds on: Gaussian Splatting FlexiCubes Stable Diffusion Score distillation sampling Shap-E
Closest papers by meaning
- Triplane Meets Gaussian Splatting: Fast and Generalizable Single-View 3D Reconstruction with Transformers (CVPR 2024 · similarity 0.92)
- Lightweight Predictive 3D Gaussian Splats (ICLR 2025 · similarity 0.91)
- Speedy-Splat: Fast 3D Gaussian Splatting with Sparse Pixels and Sparse Primitives (CVPR 2025 · similarity 0.91)
- DiffSplat: Repurposing Image Diffusion Models for Scalable Gaussian Splat Generation (ICLR 2025 · similarity 0.90)
- DiffGS: Functional Gaussian Splatting Diffusion (NeurIPS 2024 · similarity 0.90)
- UVGS: Reimagining Unstructured 3D Gaussian Splatting using UV Mapping (CVPR 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.