Research map / 3D vision / 3D generation
Real3D: The Curious Case of Neural Scene Degeneration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Real3D combines a NeRF-style implicit renderer and a DMTET tetrahedral explicit renderer with a discriminator, trained with score distillation sampling plus adversarial loss, to avoid "neural scene degeneration" in text-to-3D generation.
Problem
Text-to-3D generation via pre-trained text-to-image diffusion models often yields unrealistic, toy-like scenes that get stuck in poor local optima ("neural scene degeneration").
Approach
Render each neural scene both with an implicit NeRF renderer and an explicit DMTET mesh renderer from the same parameters, and add a discriminator (adversarial loss) alongside score distillation sampling so the scene can escape poor local optima and reach realistic rich regions.
Key results
- Real3D achieves the highest CLIP-L image-text similarity score (27.97 ± 2.91) vs. DreamFusion 25.27, Magic3D 27.50, SJC 27.21.
- In the user study, over 50% of users (52.1% official, 56.8% unofficial) ranked Real3D as best quality.
- Real3D generates a 3D scene in about 25 minutes on a single A100 GPU for 10,000 iterations.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Real3D instead of DreamFusion | Text-to-3D generation · User preference study, Official @1 | preference rank (%): 6.2 -> 52.1 | Helped |
| Use Real3D instead of DreamFusion | Text-to-3D generation · User preference study, Unofficial @1 | preference rank (%): 4.5 -> 56.8 | Helped |
| Use Real3D instead of Magic3D | Text-to-3D generation · Image-text similarity | CLIP-L score: 27.50(±3.07) -> 27.97(±2.91) | Helped |
| Add discriminator training to implicit rendering with score distillation sampling (ablation) | DreamFusion baseline · Qualitative ablation | realism | Helped |
| Add explicit rendering to implicit rendering with score distillation sampling (ablation) | DreamFusion baseline · Qualitative ablation | clarity and detail | Helped |
| Add discriminator training and explicit rendering (ablation) | DreamFusion baseline · Qualitative ablation | realism and detail | Helped |
Surprising findings
- Adversarial training alone drives scenes toward realistic regions but causes serious distortion, motivating combining it with score distillation.
Limitations
- Quality bounded by the prior knowledge network (latent Stable Diffusion), which can produce multi-face artifacts.
- Multi-face failure cases for animals and plush toys remain.
Experimental setup
Datasets: DreamFusion gallery text prompts (397 prompts, 100 sampled for user study)
Benchmarks: DreamFusion gallery prompts
Metrics: CLIP-L image-text similarity user preference ranking
Baselines: DreamFusion Magic3D SJC CLIPMesh DreamFields
Builds on: DreamFusion DMTET NeRF Magic3D score distillation sampling
Closest papers by meaning
- A 3D Scene is Worth 1K Tokens: 3D-Grounded Representation for Scene Generation at Scale (NeurIPS 2026 · similarity 0.89)
- Denoising Diffusion via Image-Based Rendering (ICLR 2024 · similarity 0.89)
- 3D-SceneDreamer: Text-Driven 3D-Consistent Scene Generation (CVPR 2024 · similarity 0.88)
- LT3SD: Latent Trees for 3D Scene Diffusion (CVPR 2025 · similarity 0.88)
- Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors (ICLR 2024 · similarity 0.88)
- SIGNeRF: Scene Integrated Generation for Neural Radiance Fields (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.