Research map / 3D vision / 3D generation
ZeroNVS: Zero-Shot 360-Degree View Synthesis from a Single Image
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces ZeroNVS, a 3D-aware diffusion model for zero-shot 360-degree single-image novel view synthesis of in-the-wild scenes, using a new viewer-centric camera parameterization and SDS anchoring to improve background diversity.
Problem
Single-image novel view synthesis for complex, multi-object in-the-wild scenes with 360-degree camera motion, where prior object-centric methods (3DoF camera conditioning, standard SDS) fail due to scale ambiguity, limited camera expressiveness, and mode collapse in backgrounds.
Approach
Replace object-centric 3DoF camera conditioning with a 6DoF+1 (relative pose plus field of view) representation whose translation scale is normalized per input view using depth quantiles from infilled depth maps, and introduce SDS anchoring, which conditions SDS on nearest DDIM-sampled anchor views instead of the input view to preserve background diversity.
Key results
- ZeroNVS achieves LPIPS 0.380 on DTU zero-shot, state of the art over DS-NeRF, PixelNeRF, SinNeRF, DietNeRF, and NeRDi, which were all trained on DTU.
- On Mip-NeRF 360 zero-shot, ZeroNVS reaches LPIPS 0.625 vs. 0.667 for Zero-1-to-3 and 0.718 for PixelNeRF, all retrained on the mixture dataset.
- In a 21-user study on Mip-NeRF 360, SDS anchoring was preferred for realism (78%), creativity (82%), and overall preference (80%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ZeroNVS instead of NeRDi | DTU zero-shot NVS · DTU | LPIPS: 0.421 -> 0.380 | Helped |
| Use ZeroNVS instead of SinNeRF | DTU zero-shot NVS · DTU | PSNR: 16.52 -> 13.55 | Hurt |
| Use ZeroNVS instead of SinNeRF | DTU zero-shot NVS · DTU | SSIM: 0.560 -> 0.469 | Hurt |
| Use ZeroNVS instead of Zero-1-to-3 | Mip-NeRF 360 zero-shot NVS · Mip-NeRF 360 Dataset | LPIPS: 0.667 -> 0.625 | Helped |
| Use ZeroNVS instead of PixelNeRF | Mip-NeRF 360 zero-shot NVS · Mip-NeRF 360 Dataset | PSNR: 16.5 -> 13.2 | Hurt |
| Use ZeroNVS instead of PixelNeRF | Mip-NeRF 360 zero-shot NVS · Mip-NeRF 360 Dataset | SSIM: 0.556 -> 0.240 | Hurt |
| Train using all three datasets rather than omitting any one dataset (ablation) | ZeroNVS · DTU | LPIPS: 0.421 vs 0.446; 0.456; 0.435 | Helped |
| Add ACID to training data (ablation) | ZeroNVS · DTU | LPIPS: 0.446 -> 0.421 | Helped |
| Add CO3D to training data (ablation) | ZeroNVS · DTU | LPIPS: 0.456 -> 0.421 | Helped |
| Add RealEstate10K to training data (ablation) | ZeroNVS · DTU | LPIPS: 0.435 -> 0.421 | Helped |
| Replace Zero-1-to-3 conditioning with viewer-centric 6DoF+1 conditioning (ablation) | ZeroNVS diffusion model · CO3D | LPIPS: 0.590 -> 0.515 | Helped |
| Replace aggregated-depth 6DoF+1 conditioning with viewer-centric conditioning (ablation) | ZeroNVS diffusion model · RealEstate10K | LPIPS: 0.417 -> 0.414 | Helped |
Surprising findings
- PSNR and SSIM do not correlate well with perceived quality in this setting, so LPIPS is weighted more heavily.
- ZeroNVS has lower PSNR (13.55) than several baselines on DTU (e.g., PixelNeRF 15.55, SinNeRF 16.52) despite the best LPIPS.
Limitations
- PSNR and SSIM are unreliable for evaluating generative novel view synthesis, and ZeroNVS's PSNR/SSIM on DTU is weaker than several baselines.
- SDS distillation is slow, making generation-based metrics such as FID costly, so diversity was assessed only via a user study.
- Ablation models were trained for fewer steps due to computational constraints.
Experimental setup
Datasets: CO3D ACID RealEstate10K DTU Mip-NeRF 360
Benchmarks: DTU Mip-NeRF 360
Metrics: LPIPS PSNR SSIM user preference
Baselines: DS-NeRF PixelNeRF SinNeRF DietNeRF NeRDi Zero-1-to-3
Builds on: Zero-1-to-3 DreamFusion Score Distillation Sampling Threestudio Mip-NeRF 360
Closest papers by meaning
- Novel View Synthesis with Pixel-Space Diffusion Models (CVPR 2025 · similarity 0.89)
- MVSplat360: Feed-Forward 360 Scene Synthesis from Sparse Views (NeurIPS 2024 · similarity 0.89)
- MOVIS: Enhancing Multi-Object Novel View Synthesis for Indoor Scenes (CVPR 2025 · similarity 0.88)
- WildCAT3D: Appearance-Aware Multi-View Diffusion in the Wild (NeurIPS 2025 · similarity 0.88)
- One View Is Enough: In-the-Wild Monocular Pretraining for Novel View Generation (NeurIPS 2026 · similarity 0.88)
- NVS-Solver: Video Diffusion Model as Zero-Shot Novel View Synthesizer (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.