Research map / 3D vision / 3D generation
MVSDet: Multi-View Indoor 3D Object Detection via Efficient Plane Sweeps
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MVSDet, a multi-view indoor 3D object detector that places pixel features into a 3D volume via a plane-sweep with probabilistic top-k depth sampling and soft weighting, regularized by pixel-aligned Gaussian Splatting rendering loss.
Problem
Multi-view indoor 3D object detection from posed images needs accurate geometry, but NeRF-based geometry (NeRF-Det) is inaccurate and plane-sweep depth estimation needs many depth planes, which is computationally expensive.
Approach
Sample only top-k depth proposals per pixel from the plane-sweep probability volume and weight each pixel feature by its normalized probability when placing it on voxels, plus a surface score multiplied into voxel features; pixel-aligned Gaussian Splatting provides a rendering loss to regularize depth.
Key results
- On ScanNet, MVSDet achieves 56.2 mAP@0.25 and 31.3 mAP@0.5 without GT geometry, versus NeRF-Det's 53.5 and 27.4.
- On ARKitScenes, MVSDet reports 42.9 mAP@0.25 and 27.0 mAP@0.5 versus NeRF-Det's 39.5 and 21.9.
- Removing probabilistic sampling drops mAP@0.25 by 19.1 points on ScanNet (ablation without rendering loss: 36.9 vs 56.0 with both components).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use MVSDet instead of NeRF-Det | multi-view indoor 3D detection · ScanNet | mAP@.5: 27.4 -> 31.3 | Helped |
| use MVSDet instead of NeRF-Det | multi-view indoor 3D detection · ARKitScenes | mAP@.5: 21.9 -> 27.0 | Helped |
| add probabilistic sampling and soft weighting (ablation) | MVSDet on ScanNet without rendering loss · ScanNet | mAP@.25: 36.9 -> 56.0 | Helped |
| add soft weighting (ablation) | MVSDet on ScanNet without rendering loss · ScanNet | mAP@.25: 50.0 -> 56.0 | Helped |
| add pixel-aligned Gaussian Splatting (ablation) | MVSDet with M=12 · ScanNet | mAP@.5: 29.7 -> 31.3 | Helped |
| add pixel-aligned Gaussian Splatting (ablation) | MVSDet with M=12 · ScanNet | RMSE: 0.674 -> 0.374 | Helped |
| increase depth planes from M=12 to M=16 (ablation) | MVSDet without Gaussian Splatting · ScanNet | mAP@.5: 29.7 -> 31.1 | Helped |
| reduce top-k depth proposals from 3 to 1 (ablation) | MVSDet · ScanNet | mAP@.25: 56.2 -> 49.3 | Hurt |
| increase top-k depth proposals from 3 to 5 (ablation) | MVSDet · ScanNet | mAP@.5: 31.3 -> 29.8 | Hurt |
Surprising findings
- Using only top-1 depth proposal severely hurts performance (49.3 vs 56.2 mAP@0.25), and MVSNet with 16 planes performs worse than the proposed method with 12 or 8 planes.
Limitations
- Feature matching fails on texture-less or reflective surfaces, as in multi-view stereo methods.
- Training memory (35 GB) and test memory (28 GB) are higher than NeRF-Det, and training time is 18 hours versus 7 for NeRF-Det.
Experimental setup
Datasets: ScanNet ARKitScenes
Benchmarks: ScanNet ARKitScenes
Metrics: mAP@0.25 mAP@0.5 RMSE
Baselines: ImGeoNet CN-RMA ImVoxelNet NeRF-Det MVSNet BEVStereo
Builds on: NeRF-Det ImVoxelNet MVSNet pixel-aligned Gaussian splatting 3D Gaussian Splatting
Closest papers by meaning
- Uncertainty Meets Diversity: A Comprehensive Active Learning Framework for Indoor 3D Object Detection (CVPR 2025 · similarity 0.86)
- Cubify Anything: Scaling Indoor 3D Object Detection (CVPR 2025 · similarity 0.86)
- UniDet3D: Multi-dataset Indoor 3D Object Detection (AAAI 2025 · similarity 0.86)
- ImOV3D: Learning Open Vocabulary Point Clouds 3D Object Detection from Only 2D Images (NeurIPS 2024 · similarity 0.85)
- 3DET-Mamba: Causal Sequence Modelling for End-to-End 3D Object Detection (NeurIPS 2024 · similarity 0.85)
- GO-N3RDet: Geometry Optimized NeRF-enhanced 3D Object Detector (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.