Research map / 3D vision / 3D generation
HarmonyView: Harmonizing Consistency and Diversity in One-Image-to-3D
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HarmonyView, a training-free diffusion sampling decomposition of multi-view guidance into separate consistency and diversity terms for single-image 3D generation, plus a CLIP-based CD score for view diversity.
Problem
Single-image 3D generation via multi-view diffusion faces a trade-off: guidance that enforces visual consistency with the input view reduces novel-view diversity, and no standard metric exists for diversity of generated views.
Approach
Decompose the multi-view diffusion score (SyncDreamer-style classifier-free guidance) into two separately weighted implicit-classifier terms: s1 pulls target views toward the input view for visual consistency, and s2 encourages diverse modes for novel views. Also proposes the reference-free CD score (CLIP-based diversity divided by semantic variance) to measure diversity.
Key results
- On GSO novel-view synthesis, HarmonyView reaches PSNR 20.69, LPIPS 0.133 and CD 0.792, versus SyncDreamer's 20.19, 0.140 and 0.717.
- On GSO 3D reconstruction, HarmonyView achieves Chamfer distance 0.0187 and volume IoU 0.6401, versus SyncDreamer's 0.0261 and 0.5421.
- In a user study on in-the-wild images, HarmonyView receives the highest CD score (0.804) and the highest Likert ratings for quality, consistency and diversity.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply HarmonyView decomposed guidance instead of SyncDreamer | Novel-view synthesis · GSO | PSNR: 20.19 -> 20.69 | Helped |
| Apply HarmonyView decomposed guidance instead of SyncDreamer | Novel-view synthesis · CD score | CD score: 0.717 -> 0.792 | Helped |
| Apply HarmonyView multi-view generation instead of SyncDreamer | 3D reconstruction · GSO | Chamfer distance: 0.0261 -> 0.0187 | Helped |
| Apply HarmonyView multi-view generation instead of SyncDreamer | 3D reconstruction · GSO | Volume IoU: 0.5421 -> 0.6401 | Helped |
| Use only s1 guidance, omitting s2 (ablation) | HarmonyView · Novel-view synthesis | PSNR: 20.69 -> 21.03 | Helped |
| Use only s1 guidance, omitting s2 (ablation) | HarmonyView · Novel-view synthesis | CD score: 0.792 -> 0.668 | Hurt |
| Use only s2 guidance, omitting s1 (ablation) | HarmonyView · Novel-view synthesis | PSNR: 20.69 -> 20.32 | Hurt |
| Use only s2 guidance, omitting s1 (ablation) | HarmonyView · Novel-view synthesis | CD score: 0.792 -> 0.764 | Hurt |
| Remove diffusion sampling guidance (ablation) | HarmonyView · Novel-view synthesis | PSNR: 20.69 -> 20.51 | No clear effect |
| Remove diffusion sampling guidance (ablation) | HarmonyView · Novel-view synthesis | CD score: 0.792 -> 0.640 | Hurt |
| Use baseline joint guidance (Eq. (3)) instead of decomposed guidance (ablation) | HarmonyView · Novel-view synthesis | CD score: 0.792 -> 0.717 | Hurt |
| Increase baseline guidance scale s from 0.5 to 1.5 (ablation) | Baseline multi-view guidance · Novel-view synthesis | PSNR: 20.55 -> 19.76 | Hurt |
Surprising findings
- Increasing the single guidance scale s in the baseline improves diversity and coherence but degrades visual consistency with the input view, showing a clear trade-off.
Limitations
- Evaluation focuses on GSO and a small set of in-the-wild images.
- The CD score relies on CLIP encoders and its alignment with human judgment is shown on a limited study.
Experimental setup
Datasets: Google Scanned Objects (GSO) Internet-collected in-the-wild images curated by SyncDreamer
Benchmarks: GSO
Metrics: PSNR SSIM LPIPS Eflow CD score Chamfer distance volumetric IoU user Likert score
Baselines: Zero123 SyncDreamer Realfusion Magic123 One-2-3-45 Point-E Shap-E
Builds on: SyncDreamer Classifier-free guidance Zero-1-to-3 NeuS
Closest papers by meaning
- Diffusion$^2$: Dynamic 3D Content Generation via Score Composition of Video and Multi-view Diffusion Models (ICLR 2025 · similarity 0.90)
- Ouroboros3D: Image-to-3D Generation via 3D-aware Recursive Diffusion (CVPR 2025 · similarity 0.90)
- SyncDreamer: Generating Multiview-consistent Images from a Single-view Image (ICLR 2024 · similarity 0.89)
- MET3R: Measuring Multi-View Consistency in Generated Images (CVPR 2025 · similarity 0.89)
- A 3D Scene is Worth 1K Tokens: 3D-Grounded Representation for Scene Generation at Scale (NeurIPS 2026 · similarity 0.89)
- Sculpt3D: Multi-View Consistent Text-to-3D Generation with Sparse 3D Prior (CVPR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.