Research map / 3D vision / Gaussian splatting
DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces DreamGaussian, a generative 3D Gaussian Splatting pipeline with SDS-based initialization, fast mesh extraction from Gaussians, and UV-space MSE texture refinement, producing textured meshes from image or text in about 2 minutes.
Problem
Optimization-based 2D-lifting 3D generation (SDS with NeRF) takes hours per sample, limiting practical use; occupancy pruning from NeRF is ineffective under ambiguous SDS supervision.
Approach
Replace NeRF with 3D Gaussian Splatting for SDS-based generation using frequent progressive densification from few Gaussians, extract a textured mesh via block-wise local density querying and back-projection, and refine the UV texture with image-space diffusion denoising and MSE loss instead of SDS.
Key results
- Image-to-3D generation in about 2 minutes with CLIP-similarity 0.738, versus 0.647 for Zero-1-to-3 (20 minutes) and 0.778 for Zero-1-to-3* (30 minutes).
- Stage 1 alone reaches CLIP-similarity 0.678 in about 1 minute.
- User study (900 valid scores) rates ours 4.31 for reference view consistency and 3.92 for overall quality, above Zero-1-to-3 (3.48, 3.11).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DreamGaussian instead of Zero-1-to-3* | Image-to-3D generation · Image-to-3D | CLIP-Similarity: 0.778 -> 0.738 | Hurt |
| Use DreamGaussian instead of Shap-E | Image-to-3D generation · Image-to-3D | Generation Time: 27 seconds -> 2 minutes | Hurt |
| Use DreamGaussian instead of Zero-1-to-3 | Image-to-3D generation · Image-to-3D | Reference View Consistency: 3.48 -> 4.31 | Helped |
| Use DreamGaussian instead of Zero-1-to-3 | Image-to-3D generation · Image-to-3D | Overall Model Quality: 3.11 -> 3.92 | Helped |
| Add the UV-space texture refinement stage (ablation) | DreamGaussian · Image-to-3D | CLIP-Similarity: 0.678 -> 0.738 | Helped |
| Add periodic densification of 3D Gaussians (ablation) | DreamGaussian stage 1 · 3D generation | Generated model quality | Helped |
| Add linear timestep annealing for the SDS loss (ablation) | DreamGaussian stage 1 · 3D generation | Generated model quality | Helped |
| Add the reference-view loss (ablation) | DreamGaussian stage 1 · Image-to-3D | Generated model quality | Helped |
| Use image-space MSE refinement instead of SDS for UV-space texture optimization | DreamGaussian texture refinement · UV-space texture refinement | Texture quality | Helped |
Surprising findings
- Occupancy pruning from NeRF is reported to be ineffective in the generative SDS setting, unlike reconstruction.
- SDS fine-tuning directly in UV texture space produces over-saturated blocky artifacts, so an MSE loss on diffusion-refined images is used instead.
Limitations
- Multi-face Janus problem, over-saturated textures, and baked lighting, shared with prior work.
- Back-view texture in image-to-3D can be blurry and lacks fine detail unless stage 2 is trained longer.
- Image-to-3D quality slightly below other optimization-based methods (CLIP-similarity and user study).
Experimental setup
Datasets: 30 images from previous works and Internet (evaluation)
Benchmarks: 30-image image-to-3D evaluation set (CLIP similarity) user study on 15 images
Metrics: CLIP-similarity generation time user rating (1-5) reference view consistency and overall qua
Baselines: Zero-1-to-3 One-2-3-45 Point-E Shap-E DreamFusion
Builds on: DreamFusion 3D Gaussian Splatting Zero-1-to-3 SDEdit Dreamtime
Closest papers by meaning
- DiffSplat: Repurposing Image Diffusion Models for Scalable Gaussian Splat Generation (ICLR 2025 · similarity 0.90)
- Generative Sparse-View Gaussian Splatting (CVPR 2025 · similarity 0.90)
- DropGaussian: Structural Regularization for Sparse-view Gaussian Splatting (CVPR 2025 · similarity 0.90)
- Text-to-3D using Gaussian Splatting (CVPR 2024 · similarity 0.89)
- DreamMesh4D: Video-to-4D Generation with Sparse-Controlled Gaussian-Mesh Hybrid Representation (NeurIPS 2024 · similarity 0.89)
- MaskGaussian: Adaptive 3D Gaussian Representation from Probabilistic Masks (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.