Research map / 3D vision / 3D generation
MOVIS: Enhancing Multi-Object Novel View Synthesis for Indoor Scenes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Extends view-conditioned diffusion novel view synthesis to multi-object indoor scenes by adding depth and object-mask inputs, an auxiliary target-view mask prediction loss, and a structure-guided timestep scheduler, plus new cross-view consistency metrics based on MASt3R matching.
Problem
Single-object view-conditioned diffusion novel view synthesizers (e.g., Zero-1-to-3) produce inconsistent object placement, shape, and appearance when applied to compositional multi-object indoor scenes, and cross-view consistency is not systematically evaluated.
Approach
Injects input-view depth and object-mask features into the denoising U-Net, jointly predicts novel-view object masks as an auxiliary task, and uses a timestep sampler whose mean decays from large to small values during training to balance global placement and fine detail learning; also proposes image-matching-based cross-view consistency metrics (Hit Rate, matching Dist.) and placement IoU.
Key results
- On the C3DFS test set, MOVIS reaches 17.432 PSNR and 58.1 placement IoU versus 14.811 PSNR and 34.4 IoU for re-trained Zero-1-to-3.
- Cross-view Hit Rate on C3DFS rises to 19.3 for MOVIS compared with 4.8 for the next best baseline (Free3D), with matching Dist. dropping from 83.6 to 44.9.
- Ablation on C3DFS shows removing the timestep scheduler drops PSNR from 17.432 to 16.166 and IoU from 58.1 to 49.1.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MOVIS instead of the strongest reported baseline | multi-object NVS · C3DFS test | PSNR: 14.811 -> 17.432 | Helped |
| Use MOVIS instead of the strongest reported baseline | multi-object NVS · Objaverse | Hit Rate: 6.1 -> 17.0 | Helped |
| Use MOVIS instead of the strongest reported baseline | multi-object NVS · Room-Texture | IoU: 19.7 -> 24.2 | Helped |
| Add input depth features (ablation) | MOVIS · C3DFS test | PSNR: 17.080 -> 17.432 | Helped |
| Add input depth features (ablation) | MOVIS · C3DFS test | IoU: 57.2 -> 58.1 | Helped |
| Add auxiliary novel-view mask prediction (ablation) | MOVIS · C3DFS test | PSNR: 16.914 -> 17.432 | Helped |
| Add auxiliary novel-view mask prediction (ablation) | MOVIS · C3DFS test | IoU: 54.7 -> 58.1 | Helped |
| Use the structure-guided timestep scheduler rather than uniform sampling (ablation) | MOVIS · C3DFS test | PSNR: 16.166 -> 17.432 | Helped |
| Use the structure-guided timestep scheduler rather than uniform sampling (ablation) | MOVIS · C3DFS test | IoU: 49.1 -> 58.1 | Helped |
Surprising findings
- Re-training Zero-1-to-3 on the multi-object C3DFS dataset did not improve its cross-view Hit Rate (1.3 vs 4.4 for original) and lowered Room-Texture PSNR, suggesting the dataset alone does not fix consistency.
Limitations
- Focuses on foreground objects and leaves background modeling to future work.
- Relies on off-the-shelf depth (Marigold) and mask (SAM) predictors at inference.
- Generalization is evaluated on limited unseen datasets; Room-Texture PSNR gain is small (10.014 vs 9.860).
Experimental setup
Datasets: C3DFS (synthetic composites from 3D-FUTURE) 3D-FUTURE
Benchmarks: C3DFS test set Room-Texture Objaverse 3D-FRONT SUNRGB-D
Metrics: PSNR SSIM LPIPS IoU Hit Rate matching Dist.
Baselines: Zero-1-to-3 Zero-1-to-3† ZeroNVS Free3D
Builds on: Zero-1-to-3 Stable Diffusion DDIM classifier guidance
Closest papers by meaning
- Novel View Synthesis with Pixel-Space Diffusion Models (CVPR 2025 · similarity 0.89)
- ViVid-1-to-3: Novel View Synthesis with Video Diffusion Models (CVPR 2024 · similarity 0.89)
- DiffuScene: Denoising Diffusion Models for Generative Indoor Scene Synthesis (CVPR 2024 · similarity 0.89)
- NAMVIS: Next-Scale Autoregressive Multi-View Image Synthesis (NeurIPS 2026 · similarity 0.89)
- Efficient-3Dim: Learning a Generalizable Single-image Novel-view Synthesizer in One Day (ICLR 2024 · similarity 0.89)
- ZeroNVS: Zero-Shot 360-Degree View Synthesis from a Single Image (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.