Research map / 3D vision / Depth and stereo
Global Latent Neural Rendering
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ConvGLR, a convolutional network that renders novel views jointly over all camera rays by processing a plane sweep volume in a low-resolution latent space, outperforming per-ray generalizable and sparse NeRF baselines.
Problem
Generalizable novel view synthesis from sparse posed input views typically learns rendering over single camera rays, treating target images as independent pixels.
Approach
Learn a global rendering operator over all camera rays jointly by treating the 5D plane sweep volume as the representation and using a convolutional encoder, depth-collapsing latent rendering block, and upsampler to render the target image in a low-resolution latent space.
Key results
- On Sparse DTU with 3 views, ConvGLR reaches 20.47 PSNR (full) versus 18.74 for PixelNeRF; ConvGLR ft reaches 25.48 with 9 views.
- On Generalizable DTU, ConvGLR achieves 31.65 PSNR and 0.080 LPIPS versus 28.50 PSNR for GPNR.
- On ILSH, ConvGLR achieves 30.17 masked PSNR, exceeding the challenge winner's masked PSNR by more than 3dB, while rendering in 0.71 s.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ConvGLR instead of the strongest listed baseline | Novel view synthesis · Sparse DTU, 3 views | PSNR: 19.55 -> 20.47 | Helped |
| Use ConvGLR instead of the strongest listed baseline | Novel view synthesis · Sparse RFF, 3 views | PSNR: 19.86 -> 19.95 | Helped |
| Use ConvGLR instead of the strongest listed baseline | Novel view synthesis · Generalizable DTU, 10 views | PSNR: 28.50 -> 31.65 | Helped |
| Use ConvGLR instead of the strongest listed baseline | Novel view synthesis · Generalizable RFF, 10 views | PSNR: 25.72 -> 26.94 | Helped |
| Use ConvGLR instead of the strongest listed baseline | Novel view synthesis · ILSH, masked | PSNR: 28.90 -> 30.17 | Helped |
| Add PSV construction rather than process concatenated input images without a PSV (ablation) | ConvGLR · Sparse DTU, 9 views | PSNR: 17.30 -> 26.33 | Helped |
| Use the PSV backbone rather than an MVS-based cost-volume backbone (ablation) | ConvGLR · Sparse DTU, 9 views | PSNR: 23.39 -> 26.33 | Helped |
| Use global latent rendering rather than RGBα planes with alpha blending (ablation) | ConvGLR · Sparse DTU, 9 views | PSNR: 24.62 -> 26.33 | Helped |
| Use 256×256 rather than 16×16 patches (ablation) | ConvGLR · Sparse DTU, 9 views | PSNR: 24.03 -> 26.33 | Helped |
| Use 256×256 rather than 32×32 patches (ablation) | ConvGLR · Sparse DTU, 9 views | PSNR: 25.79 -> 26.33 | Helped |
| Use 256×256 rather than 64×64 patches (ablation) | ConvGLR · Sparse DTU, 9 views | PSNR: 26.22 -> 26.33 | Helped |
| Use 256×256 rather than 128×128 patches (ablation) | ConvGLR · Sparse DTU, 9 views | PSNR: 26.20 -> 26.33 | Helped |
Surprising findings
- Rendering over small patches degrades sharply (16x16 patches reach 24.03 PSNR vs 26.33 for full 256x256), confirming global rendering matters.
Limitations
- Depth planes are sampled with fixed standard distributions rather than scene-adaptive sampling.
- Model is relatively large (40M parameters, 95M without parameter sharing in rendering blocks).
Experimental setup
Datasets: DTU Real Forward-Facing (RFF) LLFF IBRNet Spaces ILSH
Benchmarks: DTU sparse RFF sparse DTU generalizable RFF generalizable Spaces ILSH
Metrics: PSNR SSIM LPIPS
Baselines: SRF PixelNeRF MVSNeRF mip-NeRF DietNeRF RegNeRF MixNeRF FlipNeRF DSNeRF SparseNeRF IBRNet GPNR
Builds on: plane sweep volume VGG loss U-Net GPNR MVSNeRF
Closest papers by meaning
- UMAMI: Unifying Masked Autoregressive Models and Deterministic Rendering for View Synthesis (NeurIPS 2025 · similarity 0.87)
- GoLF-NRT: Integrating Global Context and Local Geometry for Few-Shot View Synthesis (CVPR 2025 · similarity 0.86)
- Novel View Synthesis with Pixel-Space Diffusion Models (CVPR 2025 · similarity 0.86)
- LVSM: A Large View Synthesis Model with Minimal 3D Inductive Bias (ICLR 2025 · similarity 0.85)
- WildFusion: Learning 3D-Aware Latent Diffusion Models in View Space (ICLR 2024 · similarity 0.85)
- Latent Space Imaging (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.