Research map / 3D vision / Depth and stereo
HashPoint: Accelerated Point Searching and Sampling for Neural Rendering
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HashPoint, which rasterizes point clouds into a hash table for fast point search and samples only on the primary surface along each ray, plugging into existing ray-tracing point-cloud renderers to speed them up while keeping accuracy.
Problem
Ray-tracing-based point cloud neural rendering is slow because point searching is costly and sampling is redundant across multiple surfaces, while rasterization is fast but produces holes and lower fidelity.
Approach
Project all points (not just nearest per pixel) onto the image plane and store them in a Z-order-reorganized hash table keyed by pixel for O(1) lookup with an adaptive, distance-proportional search radius; then sample candidates only on the primary (first-hit) surface using a pseudo-UDF-based volume-rendering weight.
Key results
- Integrating with Point-NeRF on Synthetic-NeRF gives 9.60 FPS versus 0.12 FPS (about 80x speedup) with PSNR 33.22 and SSIM 0.961 (vs 33.31 and 0.978 for the original).
- Integrating with Point-SLAM on Replica improves PSNR to 35.43 and SSIM to 0.983 over Point-SLAM's 35.17 and 0.975, at 1.72 FPS, 1.8x faster than depth-guided sampling and 11.5x faster than uniform multi-surface sampling.
- On Waymo, integrating with NPLF gives 1.98 FPS vs 0.33 (6x speedup) with PSNR 30.57 vs 29.96; on ShapeNet with Pointersect, 10.12 FPS vs 1.25 (8x speedup).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add HashPoint searching and primary surface sampling | Point-NeRF · Synthetic-NeRF | FPS: 0.12 -> 9.60(×80 speed up) | Helped |
| Add HashPoint searching and adaptive primary surface sampling | Point-SLAM · Replica | PSNR: 35.17 -> 35.43 | Helped |
| Add HashPoint searching and adaptive primary surface sampling | Point-SLAM · Replica | FPS: 0.95(Depth) | 0.15(US) -> 1.72 (×1.8 | ×11.5 speed up) | Helped |
| Add HashPoint searching and adaptive primary surface sampling | Point-SLAM · Replica | LPIPS: 0.124 -> 0.126 | No clear effect |
| Replace brute-force searching and K-nearest-point selection with HashPoint | NPLF · Waymo Open | FPS: 0.33 -> 1.98 (× 6 speed up) | Helped |
| Select K nearest points from primary surfaces | Pointersect · ShapeNet | PSNR: 28.0 ± 6.4 -> 29.5 ± 5.6 | Helped |
| Replace uniform-grid searching with HashPoint searching (ablation) | Point-NeRF · Synthetic-NeRF | FPS | Helped |
| Replace multi-surface sampling with adaptive primary surface sampling (ablation) | Point-NeRF · Synthetic-NeRF | FPS | Helped |
| Replace fixed-count single-surface sampling with adaptive primary surface sampling (ablation) | Point-SLAM · NeRF-Synthesis | accuracy | Helped |
| Use K-nearest-point selection instead of primary surface sampling (ablation) | Pointersect · NeRF-Synthesis | accuracy | Helped |
| Use primary surface sampling instead of K-nearest-point selection (ablation) | NPLF · Waymo | accuracy | Helped |
| Add point pruning and growing for geometry refinement (ablation) | Point-NeRF + HashPoint · NeRF-Synthesis | accuracy | Helped |
Surprising findings
- Primary-surface-only sampling alone impedes point optimization due to limited gradient propagation, so multi-surface sampling is used for the first 10K iterations before switching.
- Under noisy point clouds, K-nearest-point selection outperforms primary surface sampling, contrary to the advantage seen with correct point clouds.
Limitations
- Primary-surface-only sampling is sensitive to noisy point clouds, where K-nearest-point selection is more accurate.
- Point-NeRF integration needs geometry refinement (about 10 minutes of point pruning and growing) to work well.
- Primary-surface sampling alone limits gradient propagation and needs an initial multi-surface phase.
Experimental setup
Datasets: NeRF-Synthetic Waymo Open Replica ShapeNet sketchfab meshes
Benchmarks: NeRF-Synthetic Replica Waymo Open ShapeNet
Metrics: PSNR SSIM LPIPS FPS
Baselines: NeRF NPBG NPBG++ Huang et al. FreqPCR Point-NeRF NPLF Pointersect Point-SLAM NICE-SLAM DS-NeRF
Builds on: Point-NeRF NPLF Pointersect Point-SLAM volume rendering
Closest papers by meaning
- View Synthesis with Sculpted Neural Points (ICLR 2023 · similarity 0.88)
- NPBG++: Accelerating Neural Point-Based Graphics (CVPR 2022 · similarity 0.87)
- Pointersect: Neural Rendering With Cloud-Ray Intersection (CVPR 2023 · similarity 0.87)
- Neural Point Light Fields (CVPR 2022 · similarity 0.87)
- NeRF Director: Revisiting View Selection in Neural Volume Rendering (CVPR 2024 · similarity 0.86)
- HybridNeRF: Efficient Neural Rendering via Adaptive Volumetric Surfaces (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.