Research map / 3D vision / 3D generation

LCGen: Mining in Low-Certainty Generation for View-consistent Text-to-3D

NeurIPS 2024 · 3D vision › 3D generation · approach: Text-to-3D scene generation · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes LCGen, a plug-in loss that constrains diffusion generation certainty per viewpoint so SDS-based text-to-3D methods avoid the Janus (multi-face) problem.

Problem

The Janus Problem in SDS-based text-to-3D, where 3D models grow multiple faces/heads across views due to discrete view encoding and shared 2D diffusion priors.

Approach

Models the Janus Problem as a probability peak arising from view-independent Gaussian priors over discrete view regions, then adds an LCGen loss that uses the diffusion denoising certainty weighted by a view-based function G(c)=|phi| so different viewpoints receive different-certainty guidance; it plugs into any SDS-based method without extra data or training paradigm changes.

Key results

• On DreamFusion, Magic3D and ProlificDreamer, LCGen reduces Janus Rate from 56.67%/46.67%/63.33% to 16.67%/23.33%/20.00%.
• CLIP-Score improves for ProlificDreamer from 26.23% to 28.94% with LCGen.
• Among compared Janus-mitigation methods, LCGen on ProlificDreamer matches the lowest JR of 20.00% (tied with MVDream and DreamControl) while achieving the highest CS of 28.94%.

Measured effects reported in the paper

Surprising findings

• Using a piecewise G(c) over azimuth can still produce multiple faces, while an absolute-value G(c)=|phi| works better.
• Multi-object prompts such as 'three corgis facing different directions' still fail with LCGen and DreamControl.

Limitations

• Fails on multi-object scenes and some unusual poses (e.g., upside-down lion).
• Uses a fixed hand-designed G(c), leaving room for adaptive view-based guidance.
• Evaluated on a small set of 30 prompts with JR judged manually.

Experimental setup

Datasets: DreamFusion prompt library (30 prompts)

Benchmarks: DreamFusion prompt library

Metrics: Janus Problem Rate (JR) CLIP-Score (CS)

Baselines: DreamFusion Magic3D ProlificDreamer MVDream Perp-Neg D-SDS DreamControl

Builds on: Score Distillation Sampling DreamFusion ProlificDreamer Magic3D threestudio

Closest papers by meaning

• SweetDreamer: Aligning Geometric Priors in 2D diffusion for Consistent Text-to-3D (ICLR 2024 · similarity 0.86)
• MVDream: Multi-view Diffusion for 3D Generation (ICLR 2024 · similarity 0.86)
• Retrieval-Augmented Score Distillation for Text-to-3D Generation (ICML 2024 · similarity 0.85)
• Sculpt3D: Multi-View Consistent Text-to-3D Generation with Sparse 3D Prior (CVPR 2024 · similarity 0.85)
• Let 2D Diffusion Model Know 3D-Consistency for Robust Text-to-3D Generation (ICLR 2024 · similarity 0.85)
• Compositional 3D-aware Video Generation with LLM Director (NeurIPS 2024 · similarity 0.85)