Research map / 3D vision / 3D generation
SyncDreamer: Generating Multiview-consistent Images from a Single-view Image
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SyncDreamer, a synchronized multiview diffusion model that generates N multiview-consistent images from one view by sharing a 3D-aware feature-attention UNet initialized from Zero123.
Problem
Generating multiview-consistent images of arbitrary objects from a single view, which existing Zero123-style per-view generation fails to keep consistent in geometry and color, and distillation methods are slow.
Approach
Model the joint distribution of N target views with a shared noise predictor whose UNet (initialized from frozen Zero123) exchanges information across views at every denoising step via a 3D volume built from all noisy views and depth-wise (epipolar-style) attention on view frustum features.
Key results
- Novel view synthesis on GSO: PSNR 20.05, SSIM 0.798, LPIPS 0.146 vs. Zero123 18.93/0.779/0.166.
- Single-view reconstruction on GSO: Chamfer Distance 0.0261 and Volume IoU 0.5421, best among baselines (Zero123 0.0339 / 0.5035).
- Sampling 64 images (4 instances) takes about 40s with 50 DDIM steps on a 40G A100.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use synchronized multiview generation with 3D-aware feature attention instead of Zero123 | Zero123 · GSO NVS | PSNR: 18.93 -> 20.05 | Helped |
| Use synchronized multiview generation with 3D-aware feature attention instead of Zero123 | Zero123 · GSO SVR | Chamfer Distance: 0.0339 -> 0.0261 | Helped |
| Add 3D-aware feature attention (ablation) | SyncDreamer without 3D-aware attention · GSO multiview generation | multiview consistency | Helped |
| Initialize from Zero123 instead of Stable Diffusion (ablation) | SyncDreamer · Generalization to input images | generalization ability | Helped |
| Freeze the UNet and related Zero123 layers during training (ablation) | SyncDreamer with trainable UNet · 2D hand-drawing inputs | geometry quality | Helped |
Surprising findings
- Initializing from Stable Diffusion instead of Zero123 gave worse generalization, attributed partly to smaller batch size (192 vs 1536).
- Unfreezing the UNet during training makes the model predict input objects as thin plates, likely due to overfitting.
Limitations
- Runtime slightly longer than Zero123 due to per-step spatial feature volume construction.
- Fixed viewpoints (elevation 30 degrees) limit the generated views.
- Training UNet causes thin-plate collapse; freezing needed.
Experimental setup
Datasets: Objaverse Google Scanned Objects
Benchmarks: Google Scanned Objects
Metrics: PSNR SSIM LPIPS COLMAP reconstructed point count Chamfer Distance Volume IoU
Baselines: Zero123 RealFusion Magic123 One-2-3-45 Point-E Shap-E
Builds on: Zero123 DDPM Stable Diffusion epipolar attention NeuS
Closest papers by meaning
- HarmonyView: Harmonizing Consistency and Diversity in One-Image-to-3D (CVPR 2024 · similarity 0.89)
- SynCamMaster: Synchronizing Multi-Camera Video Generation from Diverse Viewpoints (ICLR 2025 · similarity 0.89)
- MVDream: Multi-view Diffusion for 3D Generation (ICLR 2024 · similarity 0.89)
- MET3R: Measuring Multi-View Consistency in Generated Images (CVPR 2025 · similarity 0.88)
- DreamComposer: Controllable 3D Object Generation via Multi-View Conditions (CVPR 2024 · similarity 0.88)
- Cycle3D: High-quality and Consistent Image-to-3D Generation via Generation-Reconstruction Cycle (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.