Research map / 3D vision / 3D generation
ViVid-1-to-3: Novel View Synthesis with Video Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Training-free novel-view synthesis that combines Zero-1-to-3 XL with the ZeroScope video diffusion model by denoising a smooth camera-trajectory video with both models' noise estimates, using a linearly decaying video weight.
Problem
Single-image novel view synthesis with diffusion models lacks multi-view consistency and accurate camera pose adherence, and fixes such as retraining or 3D distillation are costly or yield blurry results.
Approach
Reformulate novel-view synthesis as generating a scanning video whose camera trajectory (Slerp from the input view to the target) ends at the target view, and denoise all frames with a weighted sum of a view-conditioned image diffusion model's and a pretrained video diffusion model's noise estimates, with the video weight decaying linearly from 1.0 to 0.5.
Key results
- Achieves PSNR 24.05, SSIM 0.917, LPIPS 0.099, FOR8 0.178 and FOR16 0.060 across all views, versus 23.47 / 0.909 / 0.101 / 0.203 / 0.068 for Zero-1-to-3 XL.
- Flow-outlier ratio (FOR) improvement over Zero-1-to-3 XL is largest when the requested viewing angle is far from the input view.
- Make-It-3D scores substantially worse on image metrics (PSNR 17.13, FOR8 0.876), showing explicit 3D distillation can blur results.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Combine view-conditioned and video diffusion guidance | Zero-1-to-3 XL novel-view synthesis · GSO, all views | FOR8: 0.203 -> 0.178 | Helped |
| Keep λvideo at 1.0 through timestep 50 instead of decaying to 0.5 (ablation) | ViVid-1-to-3 · GSO, azimuth range 30–45 degrees | FOR8: 0.2923 -> 0.2958 | Hurt |
| Decay λvideo from 1.0 to 0.0 instead of 0.5 (ablation) | ViVid-1-to-3 · GSO, azimuth range 30–45 degrees | FOR8: 0.2923 -> 0.3082 | Hurt |
| Increase starting λvideo from 1.0 to 1.5 (ablation) | ViVid-1-to-3 · GSO, azimuth range 30–45 degrees | FOR8: 0.2923 -> 0.2952 | Hurt |
Surprising findings
- Explicit 3D methods (Make-It-3D, DreamGaussian-style pipelines) produced blurrier and less input-faithful renderings than pure 2D diffusion methods, attributed to monocular depth and pose inconsistencies of 2D models.
- Standard image metrics (PSNR, SSIM, LPIPS) can disagree with visual accuracy under small misalignments, motivating the optical-flow metric.
Limitations
- Still lacks an explicit 3D model and can be multi-view inconsistent.
- Large video weight causes over-smoothing and loss of detail; hyperparameters must be tuned.
- Evaluation limited to 100 GSO shapes and a limited set of peer-reviewed baselines due to compute constraints.
Experimental setup
Datasets: Google Scanned Objects (GSO)
Benchmarks: Google Scanned Objects (GSO) 100-shape rendering set
Metrics: PSNR SSIM LPIPS FOR8 FOR16
Baselines: Zero-1-to-3 XL Zero-1-to-3 SyncDreamer Magic123 Make-It-3D One-2-3-45 DreamGaussian
Builds on: Zero-1-to-3 ZeroScope Latent Diffusion Models Classifier-free guidance RAFT
Closest papers by meaning
- NVS-Solver: Video Diffusion Model as Zero-Shot Novel View Synthesizer (ICLR 2025 · similarity 0.93)
- Novel View Synthesis with Pixel-Space Diffusion Models (CVPR 2025 · similarity 0.92)
- Efficient-3Dim: Learning a Generalizable Single-image Novel-view Synthesizer in One Day (ICLR 2024 · similarity 0.91)
- Reconstruct, Inpaint, Test-Time Finetune: Dynamic Novel-view Synthesis from Monocular Videos (NeurIPS 2025 · similarity 0.91)
- Vivid-ZOO: Multi-View Video Generation with Diffusion Model (NeurIPS 2024 · similarity 0.90)
- Dynamic View Synthesis as an Inverse Problem (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.