Research map / Image generation / Autoregressive and latent generation
DiffGS: Functional Gaussian Splatting Diffusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DiffGS, a latent diffusion model that generates 3D Gaussian Splatting by disentangling Gaussians into three continuous functions (probability, color, transform) and extracting arbitrary numbers of Gaussians via octree-guided sampling.
Problem
Generative modeling of 3D Gaussian Splatting is hard because 3DGS is discrete and unstructured, so standard image/voxel/video generative frameworks do not transfer directly; voxel-based approaches are costly and limit Gaussian count.
Approach
Represent 3DGS through three continuous functions (Gaussian Probability Function, Gaussian Color Function, Gaussian Transform Function), encode them in a triplane-based Gaussian VAE, train a latent diffusion model on the latent vectors, and extract Gaussians at arbitrary counts via octree-guided sampling followed by optimization on the probability field.
Key results
- Unconditional generation on ShapeNet chairs: FID-50K 35.28 vs 59.51 for GET3D and 93.02 for DiffTF.
- Gaussian VAE reconstruction on ShapeNet chairs reaches PSNR 34.01 with 350K Gaussians, versus 28.61 with 50K.
- Text-conditioned CLIP score 33.42, above DreamGaussian (29.08), LGM (32.52) and Shap·E (32.76).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use DiffGS instead of DiffTF | unconditional generation · ShapeNet airplane | FID-50K: 110.8 -> 47.03 | Helped |
| use DiffGS instead of DiffTF | unconditional generation · ShapeNet airplane | KID-50K (‰): 9.173 -> 3.436 | Helped |
| use DiffGS instead of DiffTF and GET3D | unconditional generation · ShapeNet chair | FID-50K: 59.51 -> 35.28 | Helped |
| use DiffGS instead of DiffTF and GET3D | unconditional generation · ShapeNet chair | KID-50K (‰): 2.414 -> 2.148 | Helped |
| use DiffGS instead of the strongest listed baseline | text-to-3D generation · Text-to-3D | CLIP score: 32.76 -> 33.42 | Helped |
| remove the truncation function (ablation) | DiffGS · ShapeNet chair reconstruction | PSNR: 34.01 -> 29.39 | Hurt |
| use exponential projection instead of linear projection (ablation) | DiffGS · ShapeNet chair reconstruction | PSNR: 34.01 -> 29.74 | Hurt |
| remove Gaussian extraction optimization (ablation) | DiffGS · ShapeNet chair reconstruction | PSNR: 34.01 -> 30.34 | Hurt |
| use 50K instead of 350K Gaussians (ablation) | DiffGS · ShapeNet chair reconstruction | PSNR: 34.01 -> 28.61 | Hurt |
| use 100K instead of 350K Gaussians (ablation) | DiffGS · ShapeNet chair reconstruction | PSNR: 34.01 -> 30.35 | Hurt |
| set octree depth to 7 instead of 10 (ablation) | DiffGS · ShapeNet chair reconstruction | PSNR: 34.01 -> 29.77 | Hurt |
| set octree depth to 8 instead of 10 (ablation) | DiffGS · ShapeNet chair reconstruction | PSNR: 34.01 -> 31.70 | Hurt |
Surprising findings
- Octree sampling depth and Gaussian count improve reconstruction quality, but the paper notes text-to-3D comparisons are not like-for-like since DiffGS trains on ShapeNet while baselines use Objaverse.
Limitations
- Generated shapes sometimes have overly creative, unrealistic color schemes.
- Trained only on ShapeNet classes and a small DeepFashion3D set, so text/image comparisons with Objaverse-trained baselines are not directly fair.
- Training is costly (eight 3090 GPUs, about 5 days per class).
Experimental setup
Datasets: ShapeNet DeepFashion3D
Benchmarks: ShapeNet DeepFashion3D
Metrics: FID-50K KID-50K PSNR SSIM LPIPS CLIP score
Baselines: GET3D DiffTF DreamGaussian Shap·E LGM SplatterImage TriplaneGaussian SSDNeRF EG3D GaussianCube
Builds on: 3D Gaussian Splatting DDPM DiffusionSDF CLIP
Closest papers by meaning
- FlashGS: Efficient 3D Gaussian Splatting for Large-scale and High-resolution Rendering (CVPR 2025 · similarity 0.92)
- UVGS: Reimagining Unstructured 3D Gaussian Splatting using UV Mapping (CVPR 2025 · similarity 0.92)
- GaussianPro: 3D Gaussian Splatting with Progressive Propagation (ICML 2024 · similarity 0.92)
- Optimized Minimal 3D Gaussian Splatting (NeurIPS 2025 · similarity 0.91)
- DiffSplat: Repurposing Image Diffusion Models for Scalable Gaussian Splat Generation (ICLR 2025 · similarity 0.91)
- Fully Explicit Dynamic Gaussian Splatting (NeurIPS 2024 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.