Research map / 3D vision / NeRF and radiance fields
Inverse Rendering of Glossy Objects via the Neural Plenoptic Function and Radiance Fields
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a Neural Plenoptic Function (NeP) built from NeRF fields and ray tracing, plus material-aware cone sampling with pre-filtered environment radiance fields, for two-stage inverse rendering of glossy objects from multi-view images.
Problem
Existing NeRF-based inverse rendering methods model illumination as a 2D environment map assuming infinite lights, so they fail to capture local light interactions from nearby objects and reconstruct glossy objects poorly.
Approach
A 5D neural plenoptic function represents global illumination by recursively ray tracing the object mesh and querying a pre-filtered Mip-NeRF environment field; material-aware cone sampling derives cone angles from predicted roughness so BRDF-lobe integration needs only a few cones (8 diffuse, 4 specular), plus a dynamic weighting loss that down-weights uncertain reflective regions during geometry
Key results
- Average Chamfer Distance on five synthetic scenes of 0.00164 for the proposed method versus 0.00322 for NeRO and 0.05523 for Nerfacto.
- Material MSE on synthetic scenes averages roughness 0.003, metallic 0.020, albedo 0.036 versus NeRO's 0.007, 0.051, 0.039.
- Material-aware cone sampling achieves competitive results with 8 diffuse and 4 specular cones versus about 256 diffuse and 128 specular rays for GGX importance sampling.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the proposed method instead of NeRO for geometry reconstruction | Ours · Synthetic objects (Bunny, Box, Beethoven, Suzanne, Nefertiti) | Chamfer Distance: Bunny: 0.00153 -> 0.00147; Box: 0.00412 -> 0.00135; Beethoven: 0.00197 -> 0.00146; Suzanne: 0.00264 -> 0.00227; Nefertit | Helped |
| Use the proposed method instead of NeRO for material estimation | Ours · Synthetic objects | Roughness, metallic, and albedo MSE: Bunny: 0.002 / 0.022 / 0.044 -> 0.002 / 0.016 / 0.022; Box: 0.003 / 0.068 / 0.056 -> 0.001 / 0.019 / 0.067; Beethoven: 0 | Mixed |
| Add dynamic weighting to the decoupled-color loss (ablation) | Ours fields learning stage · Glossy-object geometry reconstruction | Surface smoothness | Helped |
| Add the environmental field instead of predicting environmental lighting with an MLP (ablation) | Ours material learning stage · Glossy-object material estimation | Material fidelity | Helped |
| Add material-aware cone sampling instead of using a fixed number of rays (ablation) | Ours material learning stage · Glossy-object material estimation | Material fidelity | Helped |
Limitations
- Material learning depends heavily on the geometry quality from the first stage.
- Decomposition between lighting, geometry, and material is not fully unambiguous; geometry color can leak into albedo.
- Geometry-dependent material ambiguity remains even with explicit lighting representation.
Experimental setup
Datasets: proposed real-world glossy object dataset proposed synthetic glossy object dataset
Benchmarks: proposed synthetic glossy dataset proposed real-world glossy dataset
Metrics: Chamfer Distance MSE
Baselines: Nerfacto NeuS NeRO
Builds on: NeRF Mip-NeRF NeuS NeRO Ref-NeuS
Closest papers by meaning
- PBR-NeRF: Inverse Rendering with Physics-Based Neural Fields (CVPR 2025 · similarity 0.91)
- Factored-NeuS: Reconstructing Surfaces, Illumination, and Materials of Possibly Glossy Objects (CVPR 2025 · similarity 0.91)
- NeISF: Neural Incident Stokes Field for Geometry and Material Estimation (CVPR 2024 · similarity 0.90)
- A General Implicit Framework for Fast NeRF Composition and Rendering (AAAI 2024 · similarity 0.89)
- A Pre-convolved Representation for Plug-and-Play Neural Illumination Fields (AAAI 2024 · similarity 0.89)
- Eclipse: Disambiguating Illumination and Materials using Unintended Shadows (CVPR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.