Research map / 3D vision / 3D generation
LucidDreamer: Towards High-Fidelity Text-to-3D Generation via Interval Score Matching
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Interval Score Matching (ISM), an SDS alternative using DDIM-inversion trajectories and interval score differences, combined with 3D Gaussian Splatting and point-cloud initialization to produce high-fidelity text-to-3D assets.
Problem
Score Distillation Sampling (SDS) for text-to-3D generation produces inconsistent, low-quality pseudo-ground-truth updates that cause over-smoothing and over-saturation in 3D assets.
Approach
Replace the single-step SDS pseudo-ground-truth with an interval score between a DDIM-inverted unconditional latent and a conditionally denoised latent, dropping a small bias term for efficiency; couple it with 3D Gaussian Splatting and a point-cloud generator (Point-E) for initialization.
Key results
- In a user study over 28 prompts, LucidDreamer obtained the lowest average rank (1.25) versus DreamFusion 3.28, Magic3D 3.44, Fantasia3D 4.53, ProlificDreamer 2.37, Text2Mesh 4.76.
- GPT-4V win rates against ProlificDreamer ranged from 58% to 63% across criteria, and against DreamFusion from 76% to 88%.
- On a single A100 with view batch size 4, LucidDreamer takes about 35 minutes versus about 8 hours for ProlificDreamer and about 30 minutes for DreamFusion.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace SDS with Interval Score Matching (ISM) | text-to-3D generation with 3DGS or NeRF · text-to-3D generation | visual detail and fidelity | Helped |
| increase the interval length δT (ablation) | LucidDreamer · text-to-3D generation | detail | Hurt |
| increase the interval length δT (ablation) | LucidDreamer · text-to-3D generation | color naturalness | Helped |
| increase the interval stride δS (ablation) | LucidDreamer · text-to-3D generation | result quality | No clear effect |
| increase the interval stride δS (ablation) | LucidDreamer · text-to-3D generation | DDIM inversion computational cost | Helped |
| initialize 3D Gaussians with a generated point cloud instead of random initialization (ablation) | LucidDreamer · text-to-3D generation | geometry quality | Helped |
Surprising findings
- ISM with normal guidance scale (7.5) yields detailed results where SDS needs CFG=100 and still looks smooth, removing the need for large guidance scales.
Limitations
- The bias term eta_t is dropped for efficiency, with its effect only analyzed in the supplement.
- Interval hyperparameters delta_T and delta_S trade off smoothness against detail and require tuning.
- Point-cloud initialization relies on an external Point-E generator.
Experimental setup
Benchmarks: 28-prompt text-to-3D set (user study) GPTEval3D winning rate
Metrics: user preference ranking GPT-4V winning rate training time
Baselines: DreamFusion Magic3D Fantasia3D ProlificDreamer Text2Mesh
Builds on: DreamFusion (SDS) ProlificDreamer (VSD) 3D Gaussian Splatting DDIM inversion Point-E
Closest papers by meaning
- Retrieval-Augmented Score Distillation for Text-to-3D Generation (ICML 2024 · similarity 0.90)
- Consistent3D: Towards Consistent High-Fidelity Text-to-3D Generation with Deterministic Sampling Prior (CVPR 2024 · similarity 0.89)
- Taming Mode Collapse in Score Distillation for Text-to-3D Generation (CVPR 2024 · similarity 0.89)
- DreamPropeller: Supercharge Text-to-3D Generation with Parallel Sampling (CVPR 2024 · similarity 0.89)
- Let 2D Diffusion Model Know 3D-Consistency for Robust Text-to-3D Generation (ICLR 2024 · similarity 0.88)
- Walking the Schrödinger Bridge: A Direct Trajectory for Text-to-3D Generation (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.