Research map / 3D vision / NeRF and radiance fields
A General Implicit Framework for Fast NeRF Composition and Rendering
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a general implicit pipeline for fast NeRF composition and dynamic shadow casting using Neural Depth Fields (NeDF), an MLP that directly predicts ray-surface intersection distance so each pixel needs one NeRF query.
Problem
Real-time composition and rendering of multiple NeRF objects under arbitrary rigid transformations, including dynamic shadows, is slow with existing NeRF acceleration methods which are specialized, memory-heavy, or limited in camera motion.
Approach
Neural Depth Fields predict, for each ray in an object's local space, a tangency-relative distance mu via a multi-level (coarse/fine) classifier rather than regression, enabling direct ray-surface intersection; a deferred-shading pipeline then needs only one NeRF evaluation per pixel and shadow mapping uses the same NeDF.
Key results
- NeDF plugin saves 97.34% render time for Mip-NeRF, 96.66% for SNeRF and 90.29% for NeuS at 800x800 with outlier resampling (e.g., 0.417s vs 15.677s for Mip-NeRF).
- On the N-object dataset, the b3 scene renders in 0.240 s without shadows and 0.398 s with shadows (PSNR 25.78 without shadow resampling).
- Authors report over 30-40x speedup over vanilla NeRF and a 4.48 s render at 2000x800 versus over 130 s for vanilla NeRF, with real-time manipulation on RTX 4090 in their CUDA version.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add NeDF acceleration to Mip-NeRF | Mip-NeRF · Mip-NeRF rendering | time (s): 0.417/15.677 | Helped |
| Add NeDF acceleration to SNeRF | SNeRF · SNeRF rendering | time (s): 0.519/15.528 | Helped |
| Add NeDF acceleration to Neus | Neus · Neus rendering | time (s): 2.11/21.72 | Helped |
| Add outlier resampling (ablation) | NeDF+Mip-NeRF · Mip-NeRF rendering | time (s): 0.234/15.677 -> 0.417/15.677 | Hurt |
| Add outlier resampling (ablation) | NeDF+SNeRF · SNeRF rendering | time (s): 0.225/15.528 -> 0.519/15.528 | Hurt |
| Add outlier resampling (ablation) | NeDF+Neus · Neus rendering | time (s): 0.202/21.72 -> 2.11/21.72 | Hurt |
| Add outlier resampling (ablation) | NeDF composition · b3, b10, b20, b50, 100-ball and 1k-ball scenes | PSNR: 23.97 -> 25.78; 19.92 -> 23.32; 15.26 -> 20.02; 15.43 -> 18.67; 24.19 -> 26.54; 22.58 -> 25.26 | Helped |
| Add outlier resampling (ablation) | NeDF composition · b3, b10, b20, b50, 100-ball and 1k-ball scenes | time (s): 0.209 -> 0.240; 0.443 -> 0.658; 0.873 -> 1.678; 1.209 -> 1.881; 1.741 -> 1.887; 16.34 -> 16.63 | Hurt |
| Replace direct regression of μ with a multi-level classifier (ablation) | NeDF intersection network · Depth discontinuities during composition | depth estimation quality | Helped |
| Increase the number of residual blocks (ablation) | NeDF intersection network · NeDF quality and speed | quality | Helped |
| Increase the number of residual blocks (ablation) | NeDF intersection network · NeDF quality and speed | speed | Hurt |
Surprising findings
- Classification over depth bins outperforms direct regression of depth, which the authors attribute to depth discontinuities near edges causing floating artifacts.
- Outlier resampling improves quality at a small speed cost, and the 1k-ball scene achieves higher PSNR than smaller scenes due to simpler geometry.
Limitations
- Limited to solid opaque surface objects; cannot represent semi-transparent weight distributions.
- Shading uses only the primary ray-surface intersection color, with no global illumination.
- Final quality is degraded relative to naive NeRF rendering, with a single-sample color approximation for previewing.
Experimental setup
Datasets: Neus real-world object dataset NeRF synthetic dataset N-object dataset (Blender-created objects)
Benchmarks: N-object dataset (b3, b10, b20, b50, 100-ball, 1k-ball) NeRF synthetic
Metrics: PSNR time (s) saved time percentage
Baselines: Mip-NeRF SNeRF NeuS NSVF DONeRF CC-NeRF vanilla NeRF
Builds on: NeRF R2L shadow mapping
Closest papers by meaning
- Rad-NeRF: Ray-decoupled Training of Neural Radiance Field (NeurIPS 2024 · similarity 0.91)
- Ced-NeRF: A Compact and Efficient Method for Dynamic Neural Radiance Fields (AAAI 2024 · similarity 0.90)
- Aleth-NeRF: Illumination Adaptive NeRF with Concealing Field Assumption (AAAI 2024 · similarity 0.90)
- A Pre-convolved Representation for Plug-and-Play Neural Illumination Fields (AAAI 2024 · similarity 0.90)
- Inverse Rendering of Glossy Objects via the Neural Plenoptic Function and Radiance Fields (CVPR 2024 · similarity 0.89)
- NeRF On-the-go: Exploiting Uncertainty for Distractor-free NeRFs in the Wild (CVPR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.