Research map / Image generation / Text-to-image diffusion
DreamFlow: High-quality text-to-3D generation by Approximating Probability Flow
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DreamFlow, a text-to-3D method that approximates the probability flow ODE of a Schrödinger Bridge with a predetermined decreasing timestep schedule (APFO), enabling faster, higher-fidelity 3D generation in three coarse-to-fine stages.
Problem
Score distillation methods for text-to-3D use random diffusion timesteps, which yields high-variance gradients, unstable optimization, and long optimization times that limit scaling to high-resolution diffusion priors.
Approach
Frame text-to-3D optimization as a multi-view Schrödinger Bridge image-to-image translation and approximate its probability flow ODE with a decreasing noise schedule, using a fine-tuned LoRA diffusion model to estimate the bridge score, replacing SDS/VSD's random-timestep objective.
Key results
- Human preference study (600 ratings) shows DreamFlow wins on photorealism against DreamFusion, Magic3D, and ProlificDreamer; DreamFusion remains better on consistency and prompt fidelity per the text.
- Outperforms ProlificDreamer on CLIP R-precision for NeRF generation and mesh fine-tuning (ViT-L/14, ViT-B/16, ViT-B/32).
- Takes about 2 hours per 3D asset on a single A100, versus ProlificDreamer's 5 hours for NeRF plus 7 hours for mesh tuning, described as 5x faster.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use DreamFlow instead of prior text-to-3D methods | Text-to-3D generation · Text-to-3D generation | generation speed: 5 times faster | Helped |
| use DreamFlow instead of DreamFusion, Magic3D, or ProlificDreamer | Text-to-3D generation · DreamFusion gallery prompts | photorealism | Helped |
| use DreamFlow instead of ProlificDreamer | NeRF generation and 3D mesh fine-tuning · DreamFusion gallery prompts | CLIP R-precision | Helped |
| do not train the auxiliary camera pose embedding (ablation) | DreamFlow · Text-to-3D generation | generation quality | No clear effect |
| remove fine-tuning of D_phi during geometry tuning (ablation) | DreamFlow mesh geometry tuning · 3D mesh fine-tuning | geometry quality | No clear effect |
| add sparsity regularization (ablation) | DreamFlow NeRF optimization · NeRF optimization | geometry quality | Hurt |
| add shading (ablation) | DreamFlow NeRF optimization · NeRF optimization | texture learning | Hurt |
Surprising findings
- The authors report that the APFO-based 2D experiment produces sharper images in 100-200 steps than VSD with 500 steps, and that simply annealing VSD's timestep does not remove blur.
Limitations
- Relies on pre-trained 2D diffusion priors that lack 3D understanding, so results may be unsatisfactory in some cases.
- Inherits biases of the pre-trained diffusion model.
- Human preference study relies on baseline results taken from papers or websites and a limited sample of 20 models per baseline.
Experimental setup
Datasets: DreamFusion gallery text prompts
Benchmarks: CLIP R-precision on DreamFusion gallery prompts human user preference study
Metrics: CLIP R-precision human preference (photorealism, 3D consistency, prompt fidel wall-clock time
Baselines: DreamFusion Magic3D ProlificDreamer
Builds on: DreamFusion ProlificDreamer Score Distillation Sampling SDEdit Instant NGP
Closest papers by meaning
- DreamTime: An Improved Optimization Strategy for Diffusion-Guided 3D Generation (ICLR 2024 · similarity 0.92)
- DreamPropeller: Supercharge Text-to-3D Generation with Parallel Sampling (CVPR 2024 · similarity 0.90)
- Instant3D: Fast Text-to-3D with Sparse-view Generation and Large Reconstruction Model (ICLR 2024 · similarity 0.89)
- Consistent Flow Distillation for Text-to-3D Generation (ICLR 2025 · similarity 0.89)
- Let 2D Diffusion Model Know 3D-Consistency for Robust Text-to-3D Generation (ICLR 2024 · similarity 0.89)
- HIFA: High-fidelity Text-to-3D Generation with Advanced Diffusion Guidance (ICLR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.