Research map / Image generation / Autoregressive and latent generation
Atlas Gaussians Diffusion for 3D Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Atlas Gaussians, a patch-based UV-sampled 3D Gaussian representation decoded by a transformer VAE, and trains a latent diffusion model on its latent space for feed-forward 3D generation.
Problem
Designing a high-fidelity and efficient 3D representation that links a VAE latent space to 3D Gaussian splatting for feed-forward latent-diffusion 3D generation, with enough Gaussians for high-quality detail.
Approach
Represent a shape as a union of learnable local patches, each parameterized by corner features in a UV square; decode any number of 3D Gaussians by sampling UV points with a learned feature-based interpolation weight, so adding Gaussians adds no network parameters. Geometry and appearance use separate decoder branches, and self-attention is localized per patch.
Key results
- Unconditional ShapeNet FID@50K: 9.90 Chair, 12.15 Car, 8.09 Plane, beating LN3Diff (16.90, 17.60, 8.84).
- Text-conditioned Objaverse: CLIP score 30.66 and FID@6K 109.5, best among GVGEN, LN3Diff, LGM, Shap-E, with 4 s inference on TITAN V.
- Ablation: learned feature-based interpolation weights improve PSNR from 26.13 (linear) to 26.56; separate geometry/appearance branches matter most (25.77 when shared).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Atlas Gaussians instead of LN3Diff | Unconditional 3D generation · ShapeNet Chair | FID@50K: 16.90 -> 9.90 | Helped |
| use Atlas Gaussians instead of LN3Diff | Unconditional 3D generation · ShapeNet Car | FID@50K: 17.60 -> 12.15 | Helped |
| use Atlas Gaussians instead of LN3Diff | Unconditional 3D generation · ShapeNet Plane | FID@50K: 8.84 -> 8.09 | Helped |
| use Atlas Gaussians instead of Shap-E | Text-conditioned 3D generation · Objaverse | CLIP Score (ViT-B/32): 30.22 -> 30.66 | Helped |
| use Atlas Gaussians instead of Shap-E | Text-conditioned 3D generation · Objaverse | FID@6K: 114.5 -> 109.5 | Helped |
| use Atlas Gaussians instead of Shap-E | Text-conditioned 3D generation · Objaverse | KID@6K (%): 4.38 -> 4.04 | Helped |
| use Atlas Gaussians instead of LGM | Text-conditioned 3D generation · Objaverse | Inference Time (GPU): 6 s (TITAN V) -> 4 s (TITAN V) | Helped |
| replace learned nonlinear weights with linear weights (ablation) | Atlas Gaussians VAE · ShapeNet validation | PSNR: 26.56 -> 26.13 | Hurt |
| add separate geometry and appearance feature branches (ablation) | Atlas Gaussians VAE · ShapeNet validation | PSNR: 25.77 -> 26.56 | Helped |
| add global features to the patch feature decoder (ablation) | Atlas Gaussians VAE · ShapeNet validation | PSNR: 26.18 -> 26.56 | Helped |
| decode scale, rotation, and opacity from geometry features rather than appearance features (ablation) | Atlas Gaussians VAE · ShapeNet validation | PSNR: 26.56 -> 26.55 | No clear effect |
| increase patches from 2048 to 4096 at approximately 32K 3DGS (ablation) | Atlas Gaussians · Objaverse | LPIPS: 0.063 -> 0.061 | Helped |
Surprising findings
- Decoding opacity, scale, and rotation from geometry features performs nearly identically to decoding them from appearance features, suggesting center and color branch separation is the key factor.
Limitations
- Uses vanilla 3D Gaussian representation, which is hard for accurate geometry extraction.
- Objaverse experiments use a filtered ~18K-shape subset due to limited compute, and baselines trained on more data may have an advantage.
Experimental setup
Datasets: ShapeNet SRN ShapeNet split G-buffer Objaverse Cap3D captions
Benchmarks: ShapeNet Chair ShapeNet Car ShapeNet Plane Objaverse text-conditioned subset
Metrics: FID@50K KID@50K FID@6K KID@6K CLIP score LPIPS PSNR inference time
Baselines: EG3D GET3D DiffRF RenderDiffusion SSDNeRF LN3Diff GVGEN LGM Shap-E
Builds on: 3D Gaussian Splatting AtlasNet LN3Diff EDM latent set representation
Closest papers by meaning
- High-fidelity 3D Object Generation from Single Image with RGBN-Volume Gaussian Reconstruction Model (CVPR 2025 · similarity 0.87)
- GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Models (CVPR 2024 · similarity 0.87)
- Prometheus: 3D-Aware Latent Diffusion Models for Feed-Forward Text-to-3D Scene Generation (CVPR 2025 · similarity 0.87)
- Hierarchical Gaussian Mixture Model Splatting for Efficient and Part Controllable 3D Generation (CVPR 2025 · similarity 0.87)
- LT3SD: Latent Trees for 3D Scene Diffusion (CVPR 2025 · similarity 0.87)
- GaussTR: Foundation Model-Aligned Gaussian Transformer for Self-Supervised 3D Spatial Understanding (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.