Research map / 3D vision / 3D generation
G3DR: Generative 3D Reconstruction in ImageNet
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes G3DR, a latent-diffusion plus triplane 3D generator trained on single-view ImageNet with a kernel-based depth gradient regularization and CLIP-based novel-view supervision, yielding diverse 3D object generation from single images.
Problem
Training 3D generators from large, unaligned, single-view multi-category datasets like ImageNet, where geometry is poor and volume collapse and degenerate novel views occur.
Approach
A depth regularization that rescales NeRF volume-rendering gradients for density and color with a kernel centered on the depth-map surface, combined with CLIP feature supervision for novel views in place of adversarial training, and multi-resolution triplane sampling without extra weights.
Key results
- On ImageNet 256x256, G3DR reaches FID 13.1 vs 16.8 for VQ3D and 19.7 for 3DGP (over 22% relative improvement) and IS 151.7 vs 124.8 for 3DGP.
- Geometry: NFS 36.5 vs 18.5 (3DGP) and 19.2 (IVID); depth accuracy 0.39 vs 0.47 (3DGP).
- Ablation: removing depth regularization raises FID from 13.1 to 18.1 and depth accuracy from 0.39 to 1.38; training takes 14.5 A100 days vs 28 for 3DGP.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add depth regularization (ablation) | G3DR · ImageNet | FID: 18.1 -> 13.1 | Helped |
| add depth regularization (ablation) | G3DR · ImageNet | NFS: 25.5 -> 36.5 | Helped |
| add CLIP supervision (ablation) | G3DR · ImageNet | FID: 19.9 -> 13.1 | Helped |
| add multi-resolution sampling (ablation) | G3DR · ImageNet | FID: 14.5 -> 13.1 | Helped |
| add depth input (ablation) | G3DR · ImageNet | FID: 64.9 -> 13.1 | Helped |
| replace alternative depth supervision with depth regularization (ablation) | G3DR · ImageNet | FID: 44.8 -> 13.1 | Helped |
| increase novel-view yaw range from U(−00°, 00°) to U(−50°, 50°) (ablation) | G3DR · ImageNet | FID: 11.89 -> 17.68 | Hurt |
Surprising findings
- Removing multi-resolution sampling barely hurts geometry (NFS 36.1 vs 36.5), while removing depth regularization severely degrades geometry.
- Alternative depth supervision (Deng et al. KL-based loss) gives much worse FID (44.8) than no depth input at all (64.9 vs 44.8 ordering aside), showing naive depth losses can harm quality.
Limitations
- Geometry evaluation lacks established protocols; relies on NFS and depth accuracy against pseudo-ground-truth depth from an estimator.
- Lower FID than 2D generators such as StyleGAN-XL and ADM; not directly comparable in visual quality.
- Trained at 128x128 with a super-resolution upsampler to 256x256; 1/3 of ImageNet used for fair comparison.
Experimental setup
Datasets: ImageNet SDIP Dogs SDIP Elephants LSUN Horses
Benchmarks: ImageNet 256x256 SDIP Dogs LSUN Horses SDIP Elephants
Metrics: FID Inception Score Non-Flatness Score depth accuracy
Baselines: 3DGP IVID VQ3D EG3D EpiGRAF StyleNeRF BigGAN StyleGAN-XL ADM
Builds on: EG3D NeRF 3DGP latent diffusion model CLIP
Closest papers by meaning
- Structured 3D Latents for Scalable and Versatile 3D Generation (CVPR 2025 · similarity 0.88)
- PartCrafter: Structured 3D Mesh Generation via Compositional Latent Diffusion Transformers (NeurIPS 2025 · similarity 0.87)
- 3D-SceneDreamer: Text-Driven 3D-Consistent Scene Generation (CVPR 2024 · similarity 0.87)
- IM-3D: Iterative Multiview Diffusion and Reconstruction for High-Quality 3D Generation (ICML 2024 · similarity 0.87)
- GTR: Improving Large 3D Reconstruction Models through Geometry and Texture Refinement (ICLR 2025 · similarity 0.87)
- Ouroboros3D: Image-to-3D Generation via 3D-aware Recursive Diffusion (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.