Research map / 3D vision / Depth and stereo
MonoInstance: Enhancing Monocular Priors via Multi-view Instance Alignment for Neural Rendering and Reconstruction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MonoInstance, which estimates per-pixel uncertainty of monocular depth priors by aligning multi-view instance back-projections and measuring point density, then uses it to weight depth loss, guide ray sampling, and add an instance-mask constraint for neural rendering and reconstruction.
Problem
Monocular depth priors used in multi-view neural rendering are inaccurate and inconsistent across views, yet existing methods apply the whole depth map indiscriminately as ground-truth supervision.
Approach
Align each segmented instance's monocular depths from multiple views in a shared 3D space and treat low point density in the fused noisy point cloud as high uncertainty; use this uncertainty to weight the depth loss, guide ray sampling, and add a multi-view instance-mask constraint on nearby views.
Key results
- On ScanNet dense-view reconstruction, MonoInstance reaches F-score 0.834 versus 0.799 for H2O-SDF and 0.733 for MonoSDF.
- On DTU sparse-view reconstruction, CD improves to 1.18 from 1.30 for NeuSurf with monocular cues.
- On LLFF 3-view novel view synthesis, PSNR 20.73 and LPIPS 0.184, outperforming COR-GS (20.45, 0.196).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply MonoInstance to MonoSDF | MonoSDF · ScanNet | F-score: 0.799 -> 0.834 | Helped |
| apply MonoInstance to MonoSDF | MonoSDF · Replica | N.C.: 0.934 -> 0.937 | Helped |
| apply MonoInstance to NeuSurf | NeuSurf · DTU (15 scenes, sparse views) | CD: 1.35 -> 1.18 | Helped |
| add uniformly applied monocular cues | NeuSurf · DTU (15 scenes, sparse views) | CD: 1.35 -> 1.30 | Helped |
| apply MonoInstance to 3DGS | 3DGS · LLFF (sparse-view NVS) | PSNR: 20.45 -> 20.73 | Helped |
| add multi-view monocular uncertainty (ablation) | MonoSDF · ScanNet | F-score: 0.749 -> 0.786 | Helped |
| add uncertainty-guided ray sampling (ablation) | MonoSDF with monocular uncertainty · ScanNet | F-score: 0.786 -> 0.805 | Helped |
| add the instance mask constraint (ablation) | MonoSDF with uncertainty-guided ray sampling · ScanNet | F-score: 0.805 -> 0.834 | Helped |
| use MonoInstance with Omnidata priors | MonoSDF · ScanNet | F-score: 0.733 -> 0.825 | Helped |
| use MonoInstance with Metric3D v2 priors | MonoSDF · ScanNet | F-score: 0.749 -> 0.834 | Helped |
| use MonoInstance with GeoWizard priors | MonoSDF · ScanNet | F-score: 0.741 -> 0.829 | Helped |
Surprising findings
- Applying monocular depths indiscriminately to all pixels does not significantly improve NeuSurf, as shown by NeuSurf versus NeuSurf† (1.35 vs 1.30 CD).
Limitations
- Uncertainty estimation depends on multi-view consistent instance segmentation and background filtering, which adds pipeline complexity.
- For indoor scenes, background instances are assumed to have reliable monocular priors in textureless areas, an assumption borrowed from prior work.
Experimental setup
Datasets: ScanNet Replica DTU LLFF
Benchmarks: ScanNet Replica DTU LLFF
Metrics: Chamfer Distance F-score Normal Consistency PSNR SSIM LPIPS
Baselines: UNISURF MonoSDF HybridNeRF H2O-SDF DebSDF RS-Recon COLMAP SparseNeuS VolRecon ReTR NeuSurf UFORecon
Builds on: MonoSDF NeuSurf 3D Gaussian Splatting MaskClustering Pixel Warping
Closest papers by meaning
- One View Is Enough: In-the-Wild Monocular Pretraining for Novel View Generation (NeurIPS 2026 · similarity 0.88)
- MonoSplat: Generalizable 3D Gaussian Splatting from Monocular Depth Foundation Models (CVPR 2025 · similarity 0.87)
- Beyond Human Perception: Understanding Multi-Object World from Monocular View (CVPR 2025 · similarity 0.87)
- DepthMaster: Unified Monocular Depth Estimation for Perspective and Panoramic Images (NeurIPS 2026 · similarity 0.86)
- Depth2Pose: A Pose-Based Benchmark for Monocular Depth Estimation without Ground-Truth Depth (NeurIPS 2026 · similarity 0.86)
- MonoVLM: Monocular 3D Visual Grounding with Vision Language Models (CVPR 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.