Research map / 3D vision / NeRF and radiance fields
GO-N3RDet: Geometry Optimized NeRF-enhanced 3D Object Detector
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GO-N3RDet, an end-to-end NeRF-enhanced multi-view 3D object detector that adds positional-embedded voxel optimization, double importance sampling, and an opacity consistency module to improve voxel geometry for indoor 3D detection.
Problem
Multi-view indoor 3D object detection suffers from inaccurate voxel features and opacity estimates because image features lack 3D positional information and NeRF-based opacity is not object-focused.
Approach
Dynamically adjusts voxel projections via predicted pixel offsets with max-pooled multi-view fusion and positional encoding (PEOM), samples NeRF points by combining NeRF density with voxel-proximity density (DIS), and enforces cross-view opacity consistency with a distance-weighted opacity average (OOM).
Key results
- On ScanNet validation, GO-N3RDet-R50 reaches 56.3 mAP@0.25 and GO-N3RDet-R101* reaches 58.6, outperforming NeRF-DetS* (57.5) and ImVoxelNet (48.4).
- On ARKitScenes validation, GO-N3RDet-R50 achieves 44.7 mAP@0.25 and 21.9 mAP@0.5 versus 26.7 and 14.4 for reproduced NeRF-Det.
- Matches CN-RMA's 58.6 mAP@0.25 on ScanNet with about 14x fewer training epochs (14 vs 80+12+100) and a much smaller grid ([40,40,16] vs [192,192,80]) without TSDF supervision.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add PEOM to naive voxel feature construction (ablation) | GO-N3RDet · ScanNet | mAP@0.25: 48.4 -> 55.2 | Helped |
| add DIS to the PEOM configuration (ablation) | GO-N3RDet · ScanNet | mAP@0.25: 55.2 -> 55.3 | Helped |
| add OOM to the PEOM configuration (ablation) | GO-N3RDet · ScanNet | mAP@0.25: 48.4 -> 53.8 | Helped |
| combine PEOM, DIS, and OOM (ablation) | GO-N3RDet · ScanNet | mAP@0.25: 48.4 -> 58.6 | Helped |
| add opacity adjustment (ablation) | NeRF-Det · ScanNet | mAP@0.25: 53.1 -> 53.3(+0.2) | No clear effect |
| add opacity adjustment (ablation) | GO-N3RDet · ScanNet | mAP@0.25: 55.2 -> 58.6(+3.4) | Helped |
| add opacity adjustment (ablation) | GO-N3RDet · ScanNet | mAP@0.5: 29.4 -> 33.7(+4.3) | Helped |
| increase uniform sampling from 64 to 128 samples (ablation) | GO-N3RDet · ScanNet | mAP@0.25: 53.2 -> 53.0 | Hurt |
| replace uniform sampling with DIS at 64 samples (ablation) | GO-N3RDet · ScanNet | mAP@0.25: 53.2 -> 54.2 | Helped |
| use DIS with 128 samples instead of uniform sampling with 64 (ablation) | GO-N3RDet · ScanNet | mAP@0.25: 53.2 -> 55.3 | Helped |
Surprising findings
- Removing the opacity prediction branch from NeRF-Det only lowers mAP@0.25 by 0.2%, suggesting NeRF opacity contributes little to its detection accuracy.
Limitations
- Gains on ARKitScenes are reported only against a locally reproduced NeRF-Det baseline, as no public results exist for other methods.
- Method relies on NeRF rendering and depth supervision for the strongest variants (marked *), which adds training complexity.
Experimental setup
Datasets: ScanNet ARKitScenes
Benchmarks: ScanNet V2 ARKitScenes
Metrics: mAP@0.25 mAP@0.5
Baselines: ImVoxelNet NeRF-Det ImGeoNet NeRF-Det++ NeRF-DetS MVSDet CN-RMA
Builds on: NeRF-Det ImVoxelNet NeRF MMDetection3D
Closest papers by meaning
- NeRF-RPN: A General Framework for Object Detection in NeRFs (CVPR 2023 · similarity 0.90)
- To View Transform or Not to View Transform: NeRF-based Pre-training Perspective (ICLR 2026 · similarity 0.89)
- SANeRF-HQ: Segment Anything for NeRF in High Quality (CVPR 2024 · similarity 0.87)
- NeRF Director: Revisiting View Selection in Neural Volume Rendering (CVPR 2024 · similarity 0.87)
- CVT-xRF: Contrastive In-Voxel Transformer for 3D Consistent Radiance Fields from Sparse Inputs (CVPR 2024 · similarity 0.86)
- NeRFPrior: Learning Neural Radiance Field as a Prior for Indoor Scene Reconstruction (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.