Research map / 3D vision / NeRF and radiance fields
MuRF: Multi-Baseline Radiance Fields
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MuRF, a feed-forward sparse-view NeRF that builds a target-view-aligned frustum volume and decodes radiance with a (2+1)D CNN, working across small and large baselines.
Problem
Sparse-view novel view synthesis fails to generalize across baselines: small-baseline feed-forward methods rely on feature matching that breaks under low overlap, while large-baseline methods are data-driven and blurry.
Approach
Construct the volume in the target view frustum (not a reference input view) by sampling multi-view colors and features plus cosine similarities, then regress the radiance field with a 3D-context CNN ((2+1)D factorized) instead of point-wise MLP or ray-wise transformers.
Key results
- On DTU 3-view small baseline, MuRF achieves 28.76 PSNR vs. 27.61 for ENeRF.
- On RealEstate10K 2-view large baseline, MuRF achieves 24.20 PSNR vs. 21.38 for AttnRend.
- On LLFF, MuRF with 4 input views (25.95 PSNR) already outperforms prior 10-view methods.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| construct the volume in the target view rather than the reference input view (ablation) | MuRF · RealEstate10K large baseline | PSNR: 20.20 -> 21.29 | Helped |
| construct the volume in the target view rather than the reference input view (ablation) | MuRF · DTU small baseline | PSNR: 26.50 -> 26.89 | Helped |
| include cosine similarities in volume elements (ablation) | MuRF · RealEstate10K large baseline | PSNR: 21.06 -> 21.29 | Helped |
| include cosine similarities in volume elements (ablation) | MuRF · DTU small baseline | PSNR: 26.68 -> 26.89 | Helped |
| include sampled features in volume elements (ablation) | MuRF · RealEstate10K large baseline | PSNR: 20.91 -> 21.29 | Helped |
| include sampled features in volume elements (ablation) | MuRF · DTU small baseline | PSNR: 26.79 -> 26.89 | Helped |
| include sampled colors in volume elements (ablation) | MuRF · RealEstate10K large baseline | PSNR: 20.04 -> 21.29 | Helped |
| include sampled colors in volume elements (ablation) | MuRF · DTU small baseline | PSNR: 25.80 -> 26.89 | Helped |
| replace the (2+1)D CNN decoder with a 3D CNN (ablation) | MuRF · RealEstate10K large baseline | PSNR: 21.29 -> 21.39 | Helped |
| replace the (2+1)D CNN decoder with a 3D CNN (ablation) | MuRF · DTU small baseline | PSNR: 26.89 -> 26.86 | Hurt |
| replace the (2+1)D CNN decoder with a spatial-only 2D CNN (ablation) | MuRF · RealEstate10K large baseline | PSNR: 21.29 -> 21.06 | Hurt |
| replace the (2+1)D CNN decoder with a spatial-only 2D CNN (ablation) | MuRF · DTU small baseline | PSNR: 26.89 -> 26.67 | Hurt |
Surprising findings
- Re-trained ENeRF produced unreasonable results under the large-baseline setting unless its number of sampling points was doubled.
- Ray Transformer decoders did not consistently outperform MLP decoders, while the (2+1)D CNN was consistently better.
Limitations
- Large-scale scene-level training datasets are scarce.
- Assumes known camera parameters and static scenes.
- Large-baseline ENeRF re-training needed hyperparameter changes, so baseline comparisons depend on enhanced settings.
Experimental setup
Datasets: RealEstate10K DTU LLFF Mip-NeRF 360 IBRNet mixed dataset
Benchmarks: DTU RealEstate10K LLFF Mip-NeRF 360
Metrics: PSNR SSIM LPIPS
Baselines: ENeRF AttnRend PixelNeRF SRF IBRNet MVSNeRF GeoNeRF MatchNeRF GPNR GNT NeuRay RegNeRF
Builds on: GMFlow IBRNet MVSNeRF ENeRF NeRF
Closest papers by meaning
- Few-Shot Neural Radiance Fields under Unconstrained Illumination (AAAI 2024 · similarity 0.90)
- ZeroRF: Fast Sparse View 360deg Reconstruction with Zero Pretraining (CVPR 2024 · similarity 0.89)
- Is Vanilla MLP in Neural Radiance Field Enough for Few-shot View Synthesis? (CVPR 2024 · similarity 0.89)
- VideoRF: Rendering Dynamic Radiance Fields as 2D Feature Video Streams (CVPR 2024 · similarity 0.89)
- Learning Robust Generalizable Radiance Field with Visibility and Feature Augmented Point Representation (ICLR 2024 · similarity 0.88)
- NeRF-VPT: Learning Novel View Representations with Neural Radiance Fields via View Prompt Tuning (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.