Research map / Image generation / Autoregressive and latent generation
CAD: Photorealistic 3D Generation via Adversarial Distillation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CAD, which distills pre-trained diffusion priors into a StyleGAN2/EG3D-style triplane 3D GAN via adversarial training on multi-view samples from Zero-1-to-3 (pruned and refined), enabling photorealistic single-image+text 3D generation.
Problem
Score distillation (SDS/VSD) for 3D generation suffers from mode-seeking artifacts such as over-saturation, over-smoothing, and Janus-like multi-face issues, and lacks photorealism and diversity.
Approach
Instead of optimizing per-object NeRFs with score distillation, train a 3D-aware GAN generator whose renderings are matched to clean samples from a pre-trained view-conditioned diffusion model via adversarial loss, with camera-pose pruning, CLIP filtering, and diffusion-based distribution refinement of the samples.
Key results
- Image CLIP score (ViT-L/14) of 82.56 vs 75.61 for Magic123 and 65.71 for DreamFusion, the best among compared methods.
- Text CLIP score (ViT-L/14) of 25.32, best among methods, versus 24.51 for ProlificDreamer.
- Generates diverse 3D objects within a second after training, versus over 10 hours of re-optimization per sample for DreamFusion and ProlificDreamer.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CAD instead of Magic123 | 3D generation from a single image and prompt · CLIP Image Score, ViT-L/14 | CLIP image score: 75.61 -> 82.56 | Helped |
| Use CAD instead of Magic123 | 3D generation from a single image and prompt · CLIP Image Score, ViT-B/32 | CLIP image score: 84.02 -> 89.16 | Helped |
| Use CAD instead of DreamFusion | 3D generation from a single image and prompt · CLIP Text Score, ViT-L/14 | CLIP text score: 24.53 -> 25.32 | Helped |
| Use CAD instead of Magic123 | 3D generation from a single image and prompt · CLIP Text Score, ViT-B/32 | CLIP text score: 29.89 -> 29.81 | No clear effect |
| Add camera pose pruning (ablation) | CAD · Synthetic-NeRF Lego scene | generation quality | Helped |
| Remove camera pose pruning (ablation) | CAD · Synthetic-NeRF Lego scene | rendering quality | Hurt |
| Remove pose pruning and ADA, and increase diffusion samples from 10K to 100K (ablation) | CAD · Synthetic-NeRF Lego scene | generation quality: 10K to 100K | Hurt |
| Add distribution refinement to Zero-1-to-3 samples (ablation) | CAD · 3D generation | generation quality and diversity | Helped |
| Use original Zero-1-to-3 samples without distribution refinement (ablation) | CAD · 3D generation | generation quality and diversity | Hurt |
| Replace distribution modeling with single-mode fitting (ablation) | CAD · Synthetic-NeRF Ficus scene | rendering quality | Hurt |
| Fit a single NeRF to raw Zero-1-to-3 samples (ablation) | CAD · Synthetic-NeRF Ficus scene | rendering quality | Hurt |
| Fit a single NeRF to refined Zero-1-to-3 samples (ablation) | CAD · Synthetic-NeRF Ficus scene | rendering quality | Hurt |
Limitations
- Optimization/training speed is bottlenecked by slow volumetric rendering (~0.1 FPS for high-resolution frames).
- Only a single conditioning input (one image plus text) is considered.
- Focused on object-level synthesis; scene-level extension not explored.
Experimental setup
Datasets: Synthetic-NeRF Synthetic-NSVF Internet images text-to-image generated images
Benchmarks: Synthetic-NeRF Synthetic-NSVF
Metrics: CLIP image score (ViT-L/14, ViT-B/32) CLIP text score (ViT-L/14, ViT-B/32) user study
Baselines: DreamFusion ProlificDreamer Magic123 Zero-1-to-3
Builds on: EG3D StyleGAN2 Zero-1-to-3 DreamFusion ProlificDreamer
Closest papers by meaning
- DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Prior (ICLR 2024 · similarity 0.90)
- Acc3D: Accelerating Single Image to 3D Diffusion Models via Edge Consistency Guided Score Distillation (CVPR 2025 · similarity 0.90)
- Efficient-3Dim: Learning a Generalizable Single-image Novel-view Synthesizer in One Day (ICLR 2024 · similarity 0.89)
- IM-3D: Iterative Multiview Diffusion and Reconstruction for High-Quality 3D Generation (ICML 2024 · similarity 0.89)
- Progressive Rendering Distillation: Adapting Stable Diffusion for Instant Text-to-Mesh Generation without 3D Data (CVPR 2025 · similarity 0.89)
- Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors (ICLR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.