Research map / 3D vision / NeRF and radiance fields
Multiplane Prior Guided Few-Shot Aerial Scene Rendering
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MPNeRF, which trains a standard NeRF with pseudo-labels from a multiplane image (MPI) branch whose generator uses a SimMIM-pretrained SwinV2 encoder, to enable few-shot aerial scene rendering.
Problem
Novel view synthesis of aerial scenes from very sparse views (3 or 5 training views), where NeRF overfits and collapses and MPI produces ghosting and cropped corners.
Approach
Couple a NeRF branch with an MPI branch: the MPI branch (guided by a frozen SimMIM-pretrained SwinV2 feature extractor) renders pseudo-labels for random pixels in unseen views, and an MSE loss distills this multiplane prior into NeRF.
Key results
- With 3 views on 16 LEVIR-NVS scenes, MPNeRF reaches mean PSNR 21.72, SSIM 0.80, LPIPS 0.19, versus 15.13 / 0.20 / 0.58 for vanilla NeRF.
- With 5 views, MPNeRF reaches mean PSNR 22.65, SSIM 0.84, LPIPS 0.16, versus FreeNeRF's 19.04 / 0.44 / 0.52.
- Ablation: removing the MPI generator drops PSNR from 21.72 to 16.38; the data-efficiency study reports needing up to 63.5% of training images to match vanilla NeRF.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MPNeRF instead of DietNeRF | few-shot aerial scene rendering · LEVIR-NVS, 3 views | PSNR: 15.71 -> 21.72 | Helped |
| Use MPNeRF instead of FreeNeRF | few-shot aerial scene rendering · LEVIR-NVS, 3 views | SSIM: 0.27 -> 0.80 | Helped |
| Use MPNeRF instead of DietNeRF | few-shot aerial scene rendering · LEVIR-NVS, 3 views | LPIPS: 0.57 -> 0.19 | Helped |
| Use MPNeRF instead of FreeNeRF | few-shot aerial scene rendering · LEVIR-NVS, 5 views | PSNR: 19.04 -> 22.65 | Helped |
| Use MPNeRF instead of FreeNeRF | few-shot aerial scene rendering · LEVIR-NVS, 5 views | SSIM: 0.44 -> 0.84 | Helped |
| Use MPNeRF instead of FreeNeRF | few-shot aerial scene rendering · LEVIR-NVS, 5 views | LPIPS: 0.52 -> 0.16 | Helped |
| Train MPI concurrently with NeRF rather than beforehand (ablation) | MPNeRF · LEVIR-NVS | PSNR: 21.49 -> 21.72 | Helped |
| Add SwinV2 pre-trained weights (ablation) | MPNeRF · LEVIR-NVS | PSNR: 20.46 -> 21.72 | Helped |
| Add multi-scale feature connections (ablation) | MPNeRF · LEVIR-NVS | PSNR: 20.05 -> 21.72 | Helped |
| Add the MPI generator (ablation) | MPNeRF · LEVIR-NVS | PSNR: 16.38 -> 21.72 | Helped |
| Use depth matching instead of the proposed LMul (ablation) | MPNeRF · LEVIR-NVS | PSNR: 21.72 -> 21.34 | Hurt |
| Use ray matching instead of the proposed LMul (ablation) | MPNeRF · LEVIR-NVS | PSNR: 21.72 -> 21.11 | Hurt |
Surprising findings
- Matching depth, ray, or non-local relation between branches as extra supervision worsened results compared with the simple color-matching loss.
- FreeNeRF gets higher PSNR in some settings but its flat renderings hurt SSIM and LPIPS.
Limitations
- Guiding strategy design needs further exploration, per the authors.
- Pseudo-labels from an imperfect MPI branch can introduce noise, so extra supervision variants hurt performance.
- Evaluated only on 16 scenes from one dataset (LEVIR-NVS).
Experimental setup
Datasets: LEVIR-NVS
Benchmarks: LEVIR-NVS
Metrics: PSNR SSIM LPIPS
Baselines: NeRF Mip-NeRF InfoNeRF DietNeRF PixelNeRF RegNeRF FreeNeRF
Builds on: NeRF Multiplane Image SimMIM DietNeRF
Closest papers by meaning
- Few-Shot Neural Radiance Fields under Unconstrained Illumination (AAAI 2024 · similarity 0.87)
- FrugalNeRF: Fast Convergence for Extreme Few-shot Novel View Synthesis without Learned Priors (CVPR 2025 · similarity 0.87)
- MuRF: Multi-Baseline Radiance Fields (CVPR 2024 · similarity 0.87)
- NeRF On-the-go: Exploiting Uncertainty for Distractor-free NeRFs in the Wild (CVPR 2024 · similarity 0.87)
- AltNeRF: Learning Robust Neural Radiance Field via Alternating Depth-Pose Optimization (AAAI 2024 · similarity 0.86)
- Rad-NeRF: Ray-decoupled Training of Neural Radiance Field (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.