Research map / 3D vision / NeRF and radiance fields
Few-Shot Neural Radiance Fields under Unconstrained Illumination
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ExtremeNeRF, a few-shot NeRF that regularizes cross-view albedo consistency (from a pretrained intrinsic decomposition net plus patch-wise albedo network) with depth consistency, and introduces the NeRF Extreme and Phototourism F^3 benchmarks for few-shot novel view synthesis under varying illumination.
Problem
Novel view synthesis from very few multi-view images captured under varying, unconstrained illumination, where NeRF's view-dependent color assumptions break down.
Approach
Enforce illumination-invariant albedo consistency across input and novel views, using an offline pseudo-albedo from a pretrained intrinsic decomposition network (IIDWW) and a patch-wise albedo decomposition network trained jointly with NeRF, supported by geometric reprojection with occlusion-aware weighting and depth consistency.
Key results
- On NeRF Extreme (3 views), ExtremeNeRF reaches SSIM 0.476 / LPIPS 0.38 on Cafe, 0.62 / 0.34 on Kitchen, and 0.67 / 0.40 on Flower, the best SSIM and LPIPS in every scene.
- Ablation: removing albedo consistency drops Cafe SSIM from 0.476 to 0.414 and raises LPIPS from 0.38 to 0.42.
- On LLFF fern, ExtremeNeRF gets SSIM 0.45 and LPIPS 0.34 versus RegNeRF 0.44 / 0.35, showing comparable performance under constant lighting.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add albedo consistency (ablation) | ExtremeNeRF · NeRF Extreme — Cafe | SSIM: 0.414 -> 0.476 | Helped |
| add FIDNet guidance for albedo consistency (ablation) | ExtremeNeRF · NeRF Extreme — Cafe | SSIM: 0.413 -> 0.476 | Helped |
| replace Albedo MLP with the full FIDNet/PIDNet approach (ablation) | ExtremeNeRF · NeRF Extreme — Cafe | SSIM: 0.413 -> 0.476 | Helped |
| use IIDWW as FIDNet instead of the alternative intrinsic decomposition model (ablation) | ExtremeNeRF · NeRF Extreme — Cafe | SSIM: 0.445 -> 0.476 | Helped |
| add depth consistency (ablation) | ExtremeNeRF · NeRF Extreme — Cafe | SSIM: 0.470 -> 0.476 | Helped |
| add the visibility mask (ablation) | ExtremeNeRF · NeRF Extreme — Cafe | SSIM: 0.404 -> 0.476 | Helped |
| use 32 × 32 rather than 64 × 64 patches (ablation) | ExtremeNeRF · Phototourism F³ — Brandenburg Gate | SSIM: 0.45 -> 0.56 | Helped |
| use 32 × 32 rather than 16 × 16 patches (ablation) | ExtremeNeRF · Phototourism F³ — Brandenburg Gate | SSIM: 0.48 -> 0.56 | Helped |
| use 32 × 32 rather than 8 × 8 patches (ablation) | ExtremeNeRF · Phototourism F³ — Brandenburg Gate | SSIM: 0.50 -> 0.56 | Helped |
| replace albedo consistency with perceptual loss (ablation) | ExtremeNeRF · NeRF Extreme — Cafe | SSIM: 0.48 -> 0.42 | Hurt |
| add albedo consistency to few-shot NeRF under varying illumination | ExtremeNeRF · Phototourism F³ — Brandenburg Gate | SSIM: 0.50 -> 0.56 | Helped |
| add albedo consistency to few-shot NeRF under varying illumination | ExtremeNeRF · Phototourism F³ — Sacre Coeur | LPIPS: 0.44 -> 0.38 | Helped |
Surprising findings
- Replacing the albedo consistency with an MLP-predicted albedo gave almost no gain over no albedo consistency at all.
- The NeROIC full model diverged on frontal-facing scenes, so only the NeROIC-Geom variant was reported.
Limitations
- Relighting given sparse inputs remains unsolved.
- Significant illumination variation can produce noisy input camera poses.
- Performance degrades on object-centric scenes lacking global context, and on challenging scenes such as Tree where all methods fail.
Experimental setup
Datasets: Phototourism NeRF Extreme LLFF DTU NeROIC dataset
Benchmarks: Phototourism F^3 NeRF Extreme LLFF DTU NeROIC TV
Metrics: SSIM LPIPS Abs Rel
Baselines: NeRF-W Ha-NeRF RegNeRF FreeNeRF NeROIC PixelNeRF MVSNeRF
Builds on: NeRF RegNeRF NeRF-W IIDWW FreeNeRF
Closest papers by meaning
- Is Vanilla MLP in Neural Radiance Field Enough for Few-shot View Synthesis? (CVPR 2024 · similarity 0.91)
- NeRF-VPT: Learning Novel View Representations with Neural Radiance Fields via View Prompt Tuning (AAAI 2024 · similarity 0.91)
- NeRF On-the-go: Exploiting Uncertainty for Distractor-free NeRFs in the Wild (CVPR 2024 · similarity 0.91)
- FrugalNeRF: Fast Convergence for Extreme Few-shot Novel View Synthesis without Learned Priors (CVPR 2025 · similarity 0.90)
- A Pre-convolved Representation for Plug-and-Play Neural Illumination Fields (AAAI 2024 · similarity 0.90)
- Bright-NeRF: Brightening Neural Radiance Field with Color Restoration from Low-Light RAW Images (AAAI 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.