Research map / 3D vision / Depth and stereo
NeLF-Pro: Neural Light Field Probes for Multi-Scale Novel View Synthesis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes NeLF-Pro, a set of local light field feature probes with a vector-matrix-matrix factorization and camera-centric neighbor blending, for fast multi-scale novel view synthesis from posed images.
Problem
Fast novel view synthesis of natural scenes that vary widely in spatial extent and granularity, where global feature grids and scene contraction scale poorly to large or long-trajectory scenes.
Approach
Model the light field as spatially distributed local probes holding multi-channel spherical feature maps, factorized via VMM (shared core factors times per-probe basis factors), and query features by blending probes near the camera, giving implicit mipmap behavior.
Key results
- On the Free dataset NeLF-Pro reaches 27.02 PSNR (vs 26.32 for F2-NeRF) and on mip-NeRF360 27.27 PSNR with 16 min training on an RTX 3090.
- On KITTI-360 NVS (50% drop) NeLF-Pro reaches 22.55 PSNR / 0.201 LPIPS in 45 minutes, versus 22.07 PSNR for PNF which takes days.
- On large-scale scenes NeLF-Pro reports 22.68 PSNR on KITTI-360-big versus 21.36 for F2-NeRF-Big, with 100M parameters and 1.6 h training.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add core factor V (ablation) | NeLF-Pro · Room and Scuol | PSNR: 29.74 -> 30.20; 27.59 -> 27.84 | Helped |
| add core factor M (ablation) | NeLF-Pro · Room and Scuol | PSNR: 28.16 -> 30.20; 24.55 -> 27.84 | Helped |
| add core factors V and M (ablation) | NeLF-Pro · Room and Scuol | PSNR: 23.78 -> 30.20; 22.76 -> 27.84 | Helped |
| select neighboring factors by camera distance instead of sample-point distance (ablation) | NeLF-Pro · Room and Scuol | PSNR: 28.97 -> 30.20; 26.72 -> 27.84 | Helped |
| randomly select probe centers from input camera centers (ablation) | NeLF-Pro · Room and Scuol | PSNR: 30.24 vs 30.20; 27.81 vs 27.84 | Mixed |
| apply spherical-coordinate scaling ν (ablation) | NeLF-Pro · Room and Scuol | PSNR: 28.65 -> 30.20; 26.97 -> 27.84 | Helped |
| aggregate factors before multiplication instead of concatenating selected factors (ablation) | NeLF-Pro · Room and Scuol | PSNR: 27.68 -> 30.20; 24.78 -> 27.84 | Helped |
Surprising findings
- Randomly placing probe centers at camera positions performs about as well as the proposed FPS/K-means placement (30.24 vs 30.20 PSNR on the room scene).
- Removing the core factor V only causes a small quality loss on small scenes, while removing core factor M causes a large drop.
Limitations
- Relies on feature grids with no customized CUDA kernels, so rendering speed is not optimized.
- Probe count and factor numbers are set per dataset scale by hand.
Experimental setup
Datasets: mip-NeRF360 Free KITTI-360 BungeeNeRF City Walk YouTube video
Benchmarks: mip-NeRF360 Free KITTI-360 NVS appearance 50% BungeeNeRF (56-Leonard, Scuol, KITTI-360-big)
Metrics: PSNR SSIM LPIPS training time parameter count
Baselines: NeRF mip-NeRF PNF mip-NeRF 360 Instant-NGP F2-NeRF GridNeRF 3DGS Plenoxels DVGO NeRF++
Builds on: TensoRF mip-NeRF 360 Factor Fields F2-NeRF IBRNet
Closest papers by meaning
- A Pre-convolved Representation for Plug-and-Play Neural Illumination Fields (AAAI 2024 · similarity 0.88)
- GoLF-NRT: Integrating Global Context and Local Geometry for Few-Shot View Synthesis (CVPR 2025 · similarity 0.87)
- Few-Shot Neural Radiance Fields under Unconstrained Illumination (AAAI 2024 · similarity 0.87)
- ProbeSDF: Light Field Probes For Neural Surface Reconstruction (CVPR 2025 · similarity 0.87)
- From Pixels to Views: Learning Angular-Aware and Physics-Consistent Representations for Light Field Microscopy (NeurIPS 2025 · similarity 0.86)
- IllumiNeRF: 3D Relighting Without Inverse Rendering (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.