Research map / 3D vision / 3D generation

One View Is Enough: In-the-Wild Monocular Pretraining for Novel View Generation

NeurIPS 2026 · 3D vision › 3D generation · approach: Diffusion-based novel view synthesis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes monocular pretraining for novel view synthesis: a frozen metric depth estimator builds pseudo-target pairs from 30M unconstrained single images, with masked losses and an adversarial term, yielding a geometry-free model (OVIE) that runs at 116 FPS without depth at inference.

Problem

Monocular novel view synthesis normally requires posed multi-view data, which restricts training to a few narrow datasets and leaves models brittle under domain shift; methods avoiding multi-view data still need depth at inference.

Approach

Use a frozen monocular depth estimator to lift single images and reproject them under sampled poses to create sparse pseudo-targets; train with validity-masked reconstruction/perceptual losses and a PatchGAN adversarial loss covering disoccluded regions, so depth is only a training scaffold and inference is a single geometry-free forward pass.

Key results

• OVIE reaches 116 FPS on a single H100, over 600x faster than the fastest baseline (VIVID at 0.19 FPS).
• OVIE-ft reaches 21.9 PSNR and 0.195 LPIPS on RealEstate10K, leading all geometry-free baselines.
• Without multi-view training data, OVIE outperforms all baselines on DL3DV (PSNR 14.8, LPIPS 0.464, FID 13.6), though PSNR is only marginally above VIVID's 14.5 in its best setting.

Surprising findings

• Removing the adversarial loss raises FID far more on DL3DV (+34.2) than on RealEstate10K (+6.3), indicating it matters most out of domain.
• L2 reconstruction loss outperforms L1 and Charbonnier despite the intuition that robust losses handle erroneous pseudo-targets.

Limitations

• Performance bounded by depth estimator failures on reflective, transparent, or low-light surfaces.
• Disoccluded content is hallucinated and may differ from the true scene; limited to moderate viewpoint change, no 360-degree or rear-view synthesis.
• Output resolution fixed at training time, so arbitrary-resolution inference is not possible.

Experimental setup

Datasets: ImageNet-21K Places Open Images OSV5M RealEstate10K DL3DV

Benchmarks: RealEstate10K DL3DV

Metrics: PSNR SSIM LPIPS FID MEt3R

Baselines: GeoGPT PhotoNVS VIVID ViewCrafter Infinite Nature-Zero SHARP FlexWorld DiffusionGS

Builds on: MoGe-2 VGGT StyleGAN-T VQ-GAN LPIPS

Closest papers by meaning

• WildCAT3D: Appearance-Aware Multi-View Diffusion in the Wild (NeurIPS 2025 · similarity 0.89)
• Reconstruct, Inpaint, Test-Time Finetune: Dynamic Novel-view Synthesis from Monocular Videos (NeurIPS 2025 · similarity 0.89)
• ZeroNVS: Zero-Shot 360-Degree View Synthesis from a Single Image (CVPR 2024 · similarity 0.88)
• Free3D: Consistent Novel View Synthesis without 3D Representation (CVPR 2024 · similarity 0.88)
• NAMVIS: Next-Scale Autoregressive Multi-View Image Synthesis (NeurIPS 2026 · similarity 0.88)
• MonoInstance: Enhancing Monocular Priors via Multi-view Instance Alignment for Neural Rendering and Reconstruction (CVPR 2025 · similarity 0.88)