Research map / 3D vision / Depth and stereo
Generalizable Novel-View Synthesis using a Stereo Camera
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes StereoNeRF, a generalizable NeRF-based view synthesis method that integrates pretrained stereo matching (UniMatch) into feature extraction, depth-guided plane-sweeping, and a stereo depth loss, and introduces the StereoNVS dataset of real and synthetic stereo images.
Problem
Generalizable novel-view synthesis from unseen scenes often yields inaccurate geometry, especially in textureless regions and thin structures; existing methods do not exploit the fixed-baseline structure of stereo-camera inputs.
Approach
Integrate stereo matching into a GeoNeRF-style generalizable NeRF: a stereo feature extractor with stereo attention modules fused with stereo-correlated features from a pretrained stereo network, depth-guided plane-sweeping that builds cost volumes around stereo depth, and a stereo depth loss using stereo pseudo-ground-truth depth.
Key results
- StereoNeRF reaches 28.44 PSNR and 0.9000 SSIM on StereoNVS-Real, versus 28.01 for GeoNeRF and 26.51 for NeuRay.
- On StereoNVS-Synthetic it reaches 33.45 PSNR and 0.1056 ABS, versus 32.85 for GeoNeRF+D and 32.13 for GeoNeRF.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use stereo inputs instead of monocular inputs (ablation) | GeoNeRF baseline · StereoNVS-Real | PSNR: 28.05 -> 28.01 | No clear effect |
| Add stereo attention module (ablation) | StereoNeRF feature extractor · StereoNVS-Real | PSNR: 28.01 -> 28.21 | Helped |
| Add stereo-correlated features (ablation) | StereoNeRF feature extractor · StereoNVS-Real | PSNR: 28.21 -> 28.31 | Helped |
| Add depth-guided plane-sweeping (ablation) | StereoNeRF · StereoNVS-Real | PSNR: 28.31 -> 28.42 | Helped |
| Add stereo depth loss (ablation) | StereoNeRF · StereoNVS-Real | PSNR: 28.42 -> 28.44 | Helped |
| Use stereo inputs instead of monocular inputs (ablation) | GeoNeRF baseline · StereoNVS-Synthetic | PSNR: 32.22 -> 32.13 | No clear effect |
| Add stereo attention module (ablation) | StereoNeRF feature extractor · StereoNVS-Synthetic | PSNR: 32.13 -> 31.90 | Hurt |
| Add stereo-correlated features (ablation) | StereoNeRF feature extractor · StereoNVS-Synthetic | PSNR: 31.90 -> 32.35 | Helped |
| Add depth-guided plane-sweeping (ablation) | StereoNeRF · StereoNVS-Synthetic | PSNR: 32.35 -> 33.09 | Helped |
| Add stereo depth loss (ablation) | StereoNeRF · StereoNVS-Synthetic | PSNR: 33.09 -> 33.45 | Helped |
| Remove depth-guided plane-sweeping (ablation) | StereoNeRF · StereoNVS-Synthetic | PSNR: 33.45 -> 32.30 | Hurt |
| Use more depth planes instead of depth-guided plane-sweeping (ablation) | StereoNeRF base model · StereoNVS-Synthetic | PSNR: 32.35 -> 32.33 | No clear effect |
Surprising findings
- Simply training baselines on stereo pairs as independent monocular inputs gives similar quality to monocular training (GeoNeRF 28.01 stereo vs 28.05 baseline on real).
- GeoNeRF+D, which uses depth maps at inference, performs worse than plain GeoNeRF on StereoNVS-Real (25.65 vs 28.01 PSNR).
Limitations
- Produces blurry images and inaccurate geometry in sparse-view settings.
- Requires stereo-camera image pairs with a fixed baseline as input.
Experimental setup
Datasets: StereoNVS-Real StereoNVS-Synthetic
Benchmarks: StereoNVS-Real StereoNVS-Synthetic
Metrics: PSNR SSIM LPIPS ABS
Baselines: SRF IBRNet GeoNeRF GNT GeoNeRF+D NeuRay
Builds on: GeoNeRF UniMatch NeRF stereo attention module
Closest papers by meaning
- G-NeRF: Geometry-enhanced Novel View Synthesis from Single-View Images (CVPR 2024 · similarity 0.87)
- Unifying Correspondence Pose and NeRF for Generalized Pose-Free Novel View Synthesis (CVPR 2024 · similarity 0.87)
- MultiDiff: Consistent Novel View Synthesis from a Single Image (CVPR 2024 · similarity 0.87)
- Novel View Synthesis with Pixel-Space Diffusion Models (CVPR 2025 · similarity 0.86)
- Geometry-guided Online 3D Video Synthesis with Multi-View Temporal Consistency (CVPR 2025 · similarity 0.86)
- Pseudo-Generalized Dynamic View Synthesis from a Video (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.