Research map / Image generation / Text-to-image diffusion
DreamAlign: Dynamic Text-to-3D Optimization with Human Preference Alignment
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DreamAlign, which curates the HP3D expert-preference text-to-3D dataset, fine-tunes MVDream with LoRA via a reward-free direct 3D preference optimization (D-3DPO) loss, and adds a preference contrastive feedback loss to SDS for text-to-3D generation.
Problem
Text-to-3D generation often produces content that deviates from human aesthetic preferences (perspective, shading, appearance, style), and existing RLHF-style reward models are costly and prone to reward hacking.
Approach
A reward-free DPO-style objective (D-3DPO) applied to a LoRA-adapted multi-view diffusion model on expert-ranked multi-view image pairs, with the resulting model used as a preference-biased prior in SDS via a contrastive loss against the base multi-view model.
Key results
- DreamAlign-2D reaches ImageReward 0.29 vs 0.11 for SD-v2.1 and -0.25 for MVDream on the validation set (Table 1).
- In a 300-response user study, 60.3% of users judged DreamAlign-3D more consistent with text and 68.6% preferred it.
- DreamAlign-2D improves CLIP text score to 35.85 from MVDream's 31.90.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune MVDream on preference-ranked data with D-3DPO and LoRA | MVDream · HP3D validation set | CLIP text score: 31.90(±2.64) -> 35.85(±1.02) | Helped |
| Fine-tune MVDream on preference-ranked data with D-3DPO and LoRA | MVDream · HP3D validation set | CLIP views-image score: 73.12(±1.41) -> 78.29(±1.13) | Helped |
| Fine-tune MVDream on preference-ranked data with D-3DPO and LoRA | MVDream · HP3D validation set | CLIP image score (GT Image): 69.64(±1.27) -> 72.28(±1.15) | Helped |
| Fine-tune MVDream on preference-ranked data with D-3DPO and LoRA | MVDream · HP3D validation set | ImageReward: -0.25(±0.18) -> 0.29(±0.09) | Helped |
| Fine-tune SD-v2.1 on preference-ranked data with D-3DPO and LoRA | SD-v2.1 · HP3D validation set | CLIP text score: 33.75(±0.58) -> 35.85(±1.02) | Helped |
| Fine-tune SD-v2.1 on preference-ranked data with D-3DPO and LoRA | SD-v2.1 · HP3D validation set | CLIP views-image score: 70.28(±2.87) -> 78.29(±1.13) | Helped |
| Fine-tune SD-v2.1 on preference-ranked data with D-3DPO and LoRA | SD-v2.1 · HP3D validation set | CLIP image score (GT Image): 62.47(±3.21) -> 72.28(±1.15) | Helped |
| Fine-tune SD-v2.1 on preference-ranked data with D-3DPO and LoRA | SD-v2.1 · HP3D validation set | ImageReward: 0.11(±0.05) -> 0.29(±0.09) | Helped |
Limitations
- Small preference dataset (about 1300 entries) and evaluation on a small validation set.
- Evaluation relies on CLIP, ImageReward and a small user study; no 3D-specific preference metrics reported.
- Preference contrastive loss relies on an assumption that the DreamAlign-2D distribution reflects human preference, which is not directly verified.
Experimental setup
Datasets: HP3D Cap3D Pick-a-Pic
Benchmarks: HP3D validation set GPTEval3D prompts
Metrics: CLIP text score CLIP view image score CLIP GT image score ImageReward user preference rate
Baselines: MVDream SD-v2.1 DreamFusion ProlificDreamer Magic3D
Builds on: MVDream Diffusion DPO DreamFusion Cap3D LoRA
Closest papers by meaning
- DreamDPO: Aligning Text-to-3D Generation with Human Preferences via Direct Preference Optimization (ICML 2025 · similarity 0.93)
- MVReward: Better Aligning and Evaluating Multi-View Diffusion Models with Human Preferences (AAAI 2025 · similarity 0.89)
- Multimodal Large Language Models Make Text-to-Image Generative Models Align Better (NeurIPS 2024 · similarity 0.89)
- Learning Multi-Dimensional Human Preference for Text-to-Image Generation (CVPR 2024 · similarity 0.89)
- Diffusion Model Alignment Using Direct Preference Optimization (CVPR 2024 · similarity 0.88)
- Smoothed Preference Optimization via ReNoise Inversion for Aligning Diffusion Models with Varied Human Preferences (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.