Research map / 3D vision / NeRF and radiance fields
Geometry-aware Reconstruction and Fusion-refined Rendering for Generalizable Neural Radiance Fields
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GeFu, a coarse-to-fine generalizable NeRF with adaptive cost aggregation, spatial-view descriptor aggregation, and consistency-aware fusion of two color decoding strategies, trained only on DTU RGB images.
Problem
Generalizable NeRFs for novel view synthesis on unseen scenes struggle in occluded, reflective, and boundary regions due to inaccurate variance-based cost volumes, sub-optimal descriptors, and suboptimal color decoding.
Approach
Adaptively reweights source-view contributions in the cost volume using the coarse-stage rendered view as target, aggregates 3D spatial context into descriptors via a 3D U-Net followed by attention over views, and fuses blending and regression color predictions with weights learned from multi-view feature consistency.
Key results
- Under generalization with three source views, GeFu reaches 29.36 PSNR on DTU and 24.28 on Real Forward-facing, versus 27.61 and 23.63 for ENeRF.
- On NeRF Synthetic with three views, GeFu achieves 26.99 PSNR versus 26.17 for ENeRF without per-scene fine-tuning.
- After 15 minutes of per-scene fine-tuning, GeFu reaches 30.10 PSNR on DTU, outperforming ENeRF fine-tuned for 1 hour (28.87).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add ACA, SVA spatial and detail modules, and CAF (ablation) | GeFu baseline · DTU, 3-view | PSNR: 27.73 -> 29.36 | Helped |
| add SVA and CAF to ACA (ablation) | GeFu · DTU, 3-view | PSNR: 28.35 -> 29.36 | Helped |
| add ACA, SVA detail module, and CAF to SVA spatial module (ablation) | GeFu · DTU, 3-view | PSNR: 28.30 -> 29.36 | Helped |
| add ACA, SVA spatial module, and CAF to SVA detail module (ablation) | GeFu · DTU, 3-view | PSNR: 27.89 -> 29.36 | Helped |
| add ACA and SVA to CAF (ablation) | GeFu · DTU, 3-view | PSNR: 28.85 -> 29.36 | Helped |
| add SVA detail module and CAF to ACA + SVA spatial module (ablation) | GeFu · DTU, 3-view | PSNR: 28.53 -> 29.36 | Helped |
| add CAF to ACA + full SVA (ablation) | GeFu · DTU, 3-view | PSNR: 28.64 -> 29.36 | Helped |
| use blending and regression with CAF instead of blending only (ablation) | GeFu · DTU, 3-view | PSNR: 27.73 -> 28.85 | Helped |
| use blending and regression with CAF instead of regression only (ablation) | GeFu · DTU, 3-view | PSNR: 26.83 -> 28.85 | Helped |
| replace autoencoder refinement with CAF (ablation) | GeFu · DTU, 3-view | PSNR: 27.89 -> 28.85 | Helped |
| replace direct weighted fusion with consistency-aware fusion (ablation) | GeFu · DTU, 3-view | PSNR: 28.10 -> 28.85 | Helped |
Surprising findings
- Regression-based color decoding gives fewer artifacts at reflections and boundaries, while blending is better in most other areas, so the two strategies are complementary.
- Direct weighted fusion of the two decoded views performs worse than the consistency-based fusion, and a naive autoencoder refinement gives only limited improvement.
Limitations
- Relies on the coarse-stage rendering quality to guide adaptive weights for the cost volume.
- Per-scene fine-tuning is needed to match NeRF on some metrics; generalization-only results trail after fine-tuning settings.
Experimental setup
Datasets: DTU Real Forward-facing NeRF Synthetic
Benchmarks: DTU Real Forward-facing NeRF Synthetic
Metrics: PSNR SSIM LPIPS Abs err Acc(2) Acc(10)
Baselines: PixelNeRF IBRNet MVSNeRF NeuRay ENeRF GNT MatchNeRF NeRF
Builds on: MVSNet ENeRF IBRNet MVSNeRF NeRF
Closest papers by meaning
- Learning Robust Generalizable Radiance Field with Visibility and Feature Augmented Point Representation (ICLR 2024 · similarity 0.87)
- AniSDF: Fused-Granularity Neural Surfaces with Anisotropic Encoding for High-Fidelity 3D Reconstruction (ICLR 2025 · similarity 0.85)
- Neural Inverse Rendering from Propagating Light (CVPR 2025 · similarity 0.85)
- HyRF: Hybrid Radiance Fields for Memory-efficient and High-quality Novel View Synthesis (NeurIPS 2025 · similarity 0.85)
- GoLF-NRT: Integrating Global Context and Local Geometry for Few-Shot View Synthesis (CVPR 2025 · similarity 0.85)
- GNeRP: Gaussian-guided Neural Reconstruction of Reflective Objects with Noisy Polarization Priors (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.