Research map / Image generation / Text-to-image diffusion
MVReward: Better Aligning and Evaluating Multi-View Diffusion Models with Human Preferences
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds a 16k-comparison human preference dataset over multi-view diffusion outputs, trains MVReward (a BLIP-based multi-view reward model) as an image-to-3D metric, and proposes MVP, a reward-plus-pretraining-loss tuning strategy applied to Wonder3D and Era3D.
Problem
Automatic metrics (FID, LPIPS, CLIPScore, BLIP score) poorly match human preferences for image-to-3D evaluation of multi-view diffusion models, and text- vs image-driven comparisons and the GSO ground-truth protocol are unfair or biased.
Approach
Train a multi-view reward model on human rankings of multi-view RGB+normal assets generated from a standardized image prompt set, then use that reward both as an evaluation metric and as a loss (MVP) that is combined with the diffusion pre-training loss to fine-tune multi-view diffusion models.
Key results
- MVReward ranking has Spearman correlation 1.00 with human evaluation ranking, versus 0.61 for GSO LPIPS, 0.86 for CLIP score, and 0.04 for BLIP score.
- MVReward reaches 83.1% accuracy on the preference evaluation; removing the BLIP backbone (CLIP instead) drops it to 81.7% and removing negative samples drops it to 74.8%.
- MVP fine-tuning improves Wonder3D's human favor rate from 9.33% to 13.7% and Era3D's from 25.3% to 29.0%, with reward and pre-training losses both needed.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune Wonder3D with MVP reward loss | Wonder3D · Standardized image prompt set | MVReward reward: 1.018 -> 1.389 | Helped |
| Fine-tune Era3D with MVP reward loss | Era3D · Standardized image prompt set | MVReward reward: 1.503 -> 1.676 | Helped |
| Fine-tune Wonder3D with MVP reward loss | Wonder3D · Standardized image prompt set | LPIPS: 0.141 -> 0.142 | Hurt |
| Fine-tune Era3D with MVP reward loss | Era3D · Standardized image prompt set | LPIPS: 0.127 -> 0.124 | Helped |
| replace BLIP backbone with CLIP (ablation) | MVReward · MVReward test set | accuracy: 83.1 -> 81.7 | Hurt |
| remove multi-view self-attention (ablation) | MVReward · MVReward test set | accuracy: 83.1 -> 81.5 | Hurt |
| remove modality-reversed negative samples (ablation) | MVReward · MVReward test set | accuracy: 83.1 -> 74.8 | Hurt |
| add reward loss to pre-trained loss (ablation) | Wonder3D · Standardized image prompt set | MVReward reward: 1.018 -> 1.389 | Helped |
| add reward loss to pre-trained loss (ablation) | Era3D · Standardized image prompt set | MVReward reward: 1.503 -> 1.676 | Helped |
Surprising findings
- BLIP score ranks models almost uncorrelated with human preference (Spearman 0.04), and GSO LPIPS is only moderately aligned (0.61).
Limitations
- Reward model needs more training data for more reliable evaluation.
- Only evaluates multi-view images rather than direct 3D representations such as meshes.
- MVP fine-tuning was only applied to Wonder3D officially and to an unofficial Era3D reproduction.
Experimental setup
Datasets: Objaverse GSO DALL·E generated image prompts 16k expert pairwise comparisons (new)
Benchmarks: standardized image prompt set (600 images) GSO (LPIPS)
Metrics: Spearman correlation to human ranking human favor rate (%) pairwise evaluation accuracy win rate LPIPS CLIP score BLIP score
Baselines: Envision3D SyncDreamer Wonder3D Zero123++ Era3D CLIP score BLIP score LPIPS on GSO
Builds on: ImageReward ReFL BLIP Wonder3D Zero123++
Closest papers by meaning
- MVDream: Multi-view Diffusion for 3D Generation (ICLR 2024 · similarity 0.90)
- DreamAlign: Dynamic Text-to-3D Optimization with Human Preference Alignment (AAAI 2025 · similarity 0.89)
- Personalized Preference Fine-tuning of Diffusion Models (CVPR 2025 · similarity 0.89)
- Diffusion Model Alignment Using Direct Preference Optimization (CVPR 2024 · similarity 0.89)
- Aligning Diffusion Models by Optimizing Human Utility (NeurIPS 2024 · similarity 0.89)
- Diffusion Model as a Noise-Aware Latent Reward Model for Step-Level Preference Optimization (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.