Research map / 3D vision / Depth and stereo

Fine-Detail Monocular Geometry Estimation with Self-Guided Sparse Volumetric Refinement

NeurIPS 2026 · 3D vision › Depth and stereo · approach: Foundation models for monocular geometry

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MoGe-3, which refines a MoGe-2 point map in 3D using self-guided sparse voxel convolutions (SSR) so that feature mixing follows 3D rather than image-plane proximity, improving fine-detail monocular point-map accuracy.

Problem

Monocular geometry estimators decode 3D geometry in a 2D parameterization, so features mix across depth discontinuities and over-smooth thin or elongated structures, causing 3D distortions even when 2D depth looks plausible.

Approach

Lift the coarse point map onto a sparse voxel shell in log-depth coordinates and iteratively predict log-depth residuals with a sparse 3D U-Net whose voxel neighborhoods follow 3D locality, injecting DINOv2 2D features at the bottleneck; the voxel shell is re-built from each refined prediction.

Key results

• On local fine-detail metrics (Spring and Synth4K), MoGe-3 ViT-L reaches point delta_0.01 of 55.9 vs 46.6 for MoGe-2 and Rel^p 2.79 vs 3.19.
• Global relative point Rel^p improves to 7.91 (vs 8.73 MoGe-2) and metric depth delta_1 to 82.7 (vs 77.3 MoGe-2) with ViT-L.
• ViT-L 3-step SSR adds latency of 121 ms vs 39 ms for MoGe-2 on a single A100 in FP16; the 2D conv refiner ablation yields no local gain.

Surprising findings

• A parameter-matched 2D U-Net refiner fails to improve local geometry, while the 3D sparse refiner does, supporting the 2D-vs-3D parameterization argument.
• Boundary F1 is not a good proxy for 3D geometric quality; InfiniDepth ranks best on boundary F1 yet its 3D point maps show more distortion.

Limitations

• Regression-based paradigm struggles with pixel ambiguity at object boundaries, producing fly-points and non-sharp depth edges.
• Boundary F1 is 2D and does not capture 3D quality; method ranks third on boundary F1.
• Refiner gradients come only from synthetic data, and sparse 3D conv adds overhead relative to base model.

Experimental setup

Datasets: 3D Ken Burns ApolloSynthetic EDEN GTA-SfM Hypersim IRS MatrixCity MidAir MVS-Synth Objaverse OmniWorld Structured3D

Benchmarks: Synth4K Spring NYUv2 KITTI ETH3D DIODE Sintel iBims-1 HAMMER

Metrics: Rel delta threshold accuracy boundary F1 latency (ms)

Baselines: Depth Pro UniDepth V2 Depth Anything 3 UniK3D Pixel-Perfect Depth InfiniDepth MoGe-2

Builds on: MoGe-2 DINOv2 MoGe sparse 3D U-Net

Closest papers by meaning

• SurGe: Improved Surface Geometry in Point Maps (NeurIPS 2026 · similarity 0.88)
• MoGe-2: Accurate Monocular Geometry with Metric Scale and Sharp Details (NeurIPS 2025 · similarity 0.87)
• GeoSVR: Taming Sparse Voxels for Geometrically Accurate Surface Reconstruction (NeurIPS 2025 · similarity 0.86)
• GeoDepth: From Point-to-Depth to Plane-to-Depth Modeling for Self-Supervised Monocular Depth Estimation (CVPR 2025 · similarity 0.86)
• Fin3R: Fine-tuning Feed-forward 3D Reconstruction Models via Monocular Knowledge Distillation (NeurIPS 2025 · similarity 0.86)
• UniK3D: Universal Camera Monocular 3D Estimation (CVPR 2025 · similarity 0.86)