Research map / Image generation / Text-to-image diffusion
DreamDPO: Aligning Text-to-3D Generation with Human Preferences via Direct Preference Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DreamDPO, an SDS-style text-to-3D optimizer that builds on-the-fly noise pairs, ranks them with a reward model (HPSv2 default) or an LMM (Qwen-VL), and updates the 3D representation with a piecewise preference loss.
Problem
Text-to-3D generation often fails to align outputs with human preferences; existing reward-guided methods rely on accurate pointwise quality scores and offer limited controllability beyond quality.
Approach
Replace pointwise score-guided optimization with pairwise preference-guided optimization: two Gaussian-noised renders of the same view are ranked by a reward model or LMM, and a SimPO-inspired, reference-free piecewise loss pulls the winner and pushes the loser, with the winner-only pull used when the score gap is small.
Key results
- On GPTEval3D, DreamDPO ranks first on all five criteria and overall (overall Elo 1203.1 vs. 1097.7 for MVDream), with ImageReward -0.35, the best among the compared methods listed in the table.
- Reported gains over prior SOTA include text-asset alignment +28.4, 3D plausibility +24.4, texture details +48.4, geometry details +41.4, overall +24.4.
- Optimization takes about two hours on a single NVIDIA RTX A6000 GPU.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DreamDPO instead of the strongest baseline | Text-to-3D generation · GPTEval3D (110 prompts) | Alignment: +28.4 | Helped |
| Use DreamDPO instead of the strongest baseline | Text-to-3D generation · GPTEval3D (110 prompts) | Plausibility: +24.4 | Helped |
| Use DreamDPO instead of the strongest baseline | Text-to-3D generation · GPTEval3D (110 prompts) | Text-geometry alignment: +25.8 | Helped |
| Use DreamDPO instead of the strongest baseline | Text-to-3D generation · GPTEval3D (110 prompts) | Texture Details: +48.4 | Helped |
| Use DreamDPO instead of the strongest baseline | Text-to-3D generation · GPTEval3D (110 prompts) | Geometry Details: +41.4 | Helped |
| Use DreamDPO instead of the strongest baseline | Text-to-3D generation · GPTEval3D (110 prompts) | Overall: +24.4 | Helped |
| Use a small but non-zero score-gap threshold rather than τ = 0 (ablation) | DreamDPO · 2D toy experiments | Generation stability | Helped |
| Use different Gaussian noises at the same timestep rather than the same noise at different timesteps (ablation) | DreamDPO pairwise example construction · Pairwise example construction evaluation | Generation quality | Helped |
| Use ImageReward instead of HPSv2 as the reward model (ablation) | DreamDPO · Text-to-3D generation | Generation quality | Hurt |
| Use SD2.1 as the diffusion backbone instead of MVDream (ablation) | DreamDPO · Text-to-3D generation | Generation performance | Mixed |
| Use LMM-based ranking with explicit instruction questions (ablation) | DreamDPO · Text-to-3D generation | Text alignment | Helped |
Surprising findings
- Using different timesteps for the pair instead of different noises yields worse results, since noisy latents at different timesteps are easier to distinguish.
- Setting the score-gap threshold to zero produces chaotic gradients, so a small non-zero threshold is needed.
Limitations
- Quality is largely bounded by the generative model and AI feedback quality.
- Open API LMM judges introduce instability, so instruction prompts must be carefully designed.
- Evaluation relies on ImageReward and GPT-4V-based Elo, with no human study reported.
Experimental setup
Datasets: GPTEval3D 110 prompts
Benchmarks: GPTEval3D
Metrics: ImageReward CLIP score GPTEval3D Elo (alignment, plausibility, T-G coherency, geo d
Baselines: DreamFusion DreamGaussian Instant3D Fantasia3D Latent-NeRF Magic3D MVDream Point-E ProlificDreamer Shap-E SJC SyncDreamer
Builds on: DreamFusion MVDream SimPO Diffusion-DPO threestudio
Closest papers by meaning
- DreamAlign: Dynamic Text-to-3D Optimization with Human Preference Alignment (AAAI 2025 · similarity 0.93)
- GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generation (CVPR 2024 · similarity 0.88)
- Progressive3D: Progressively Local Editing for Text-to-3D Content Creation with Complex Semantic Prompts (ICLR 2024 · similarity 0.88)
- Layout-your-3D: Controllable and Precise 3D Generation with 2D Blueprint (ICLR 2025 · similarity 0.87)
- LucidDreamer: Towards High-Fidelity Text-to-3D Generation via Interval Score Matching (CVPR 2024 · similarity 0.86)
- MVReward: Better Aligning and Evaluating Multi-View Diffusion Models with Human Preferences (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.