Research map / 3D vision / Gaussian splatting
Text-to-3D using Gaussian Splatting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GSGEN, a text-to-3D method that represents assets with 3D Gaussian Splatting and optimizes them in two stages, geometry optimization with 2D plus Point-E 3D SDS and appearance refinement with compactness-based densification.
Problem
Text-to-3D generation with SDS and implicit representations (NeRF, DMTet) suffers from inaccurate geometry such as the Janus problem and is slow to produce detailed, colorful models, and it is hard to inject explicit 3D priors.
Approach
Use 3D Gaussians as the representation so that a point-cloud diffusion prior (Point-E) can be applied directly via a 3D SDS loss on Gaussian positions, initialize Gaussians from Point-E point clouds, and add compactness-based densification to fill gaps under SDS gradients.
Key results
- GSGEN generates 3D assets in about 40 minutes, on par with Magic3D and Fantasia3D, with improved fidelity and richer details (qualitative comparison).
- Ablations show DreamFusion-style initialization causes degenerate geometry, while Point-E initialization and the 3D prior reduce Janus-type failures.
- Compactness-based densification is an effective supplement to view-space gradient densification under SDS guidance.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| initialize Gaussian positions with Point-E-generated geometry instead of an origin-centered Gaussian distribution (ablation) | GSGEN · Text-to-3D generation, especially asymmetric prompts | geometry consistency | Helped |
| remove Point-E point-cloud guidance during geometry optimization (ablation) | GSGEN · Text-to-3D generation, especially asymmetric geometries | geometry consistency | Hurt |
| use only view-space gradient densification with Tpos = 0.0002 instead of adding compactness-based densification (ablation) | GSGEN · Text-to-3D generation | appearance detail | Mixed |
| use only view-space gradient densification with Tpos = 0.02 instead of adding compactness-based densification (ablation) | GSGEN · Text-to-3D generation | appearance detail | Hurt |
| add compactness-based densification to view-space gradient densification with Tpos = 0.02 (ablation) | GSGEN · Text-to-3D generation | appearance detail and continuity | Helped |
Surprising findings
- Random color initialization was preferred over Point-E's generated colors because direct use of generated colors was detrimental in early experiments.
Limitations
- Complex or logically intricate prompts give unsatisfying results due to limited language understanding of Point-E and the CLIP text encoder.
- The 3D prior mitigates but does not eliminate Janus and other degenerations, especially for prompts biased in the guidance models.
- Evaluation is mainly qualitative; no quantitative metrics are reported in the main text.
Experimental setup
Baselines: DreamFusion Magic3D Fantasia3D ProlificDreamer Latent-NeRF
Builds on: DreamFusion 3D Gaussian Splatting Point-E Stable Diffusion ProlificDreamer
Closest papers by meaning
- Hierarchical Gaussian Mixture Model Splatting for Efficient and Part Controllable 3D Generation (CVPR 2025 · similarity 0.91)
- GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Models (CVPR 2024 · similarity 0.90)
- DirectTriGS: Triplane-based Gaussian Splatting Field Representation for 3D Generation (CVPR 2025 · similarity 0.90)
- DiffSplat: Repurposing Image Diffusion Models for Scalable Gaussian Splat Generation (ICLR 2025 · similarity 0.89)
- Turbo3D: Ultra-fast Text-to-3D Generation (CVPR 2025 · similarity 0.89)
- DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation (ICLR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.