Research map / 3D vision / NeRF and radiance fields
Synergistic Integration of Coordinate Network and Tensorial Feature for Improving Neural Radiance Fields from Sparse Inputs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SynergyNeRF, which combines a ReLU coordinate MLP (low-frequency bias) with TensoRF/HexPlane-style multi-plane features via residual concatenation and a channel-wise curriculum, improving static and dynamic NeRFs from sparse views.
Problem
Multi-plane (grid-based) NeRF representations struggle to capture low-frequency structure and overfit high-frequency details, causing instability and inefficiency when training poses are sparse.
Approach
Residually concatenate coordinate inputs and multi-plane features into a coordinate MLP so the coordinate branch captures low-frequency content while planes capture fine details, plus a channel-wise cosine curriculum that gradually engages multi-plane channels.
Key results
- On 8-view NeRF-synthetic, the method reaches 24.56 average PSNR, the best among compared methods (K-Planes 24.24, FreeNeRF 23.40).
- On D-NeRF with 25 poses, it reaches 25.34 PSNR vs HexPlane 24.15 and K-Planes 22.68.
- A reduced 1.0M-parameter variant reaches 25.10 PSNR on D-NeRF, above full-size K-Planes (23.85) and HexPlane (24.00).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| integrate coordinate features with multi-plane features using the proposed residual concatenation | NeRF-synthetic, Ours · NeRF-synthetic, 8 views | Avg. PSNR: 23.15 -> 24.56 | Helped |
| integrate coordinate features with multi-plane features | D-NeRF, Ours · D-NeRF, 25 poses | Avg. PSNR: 24.15 -> 25.34 | Helped |
| use skip connections at every layer instead of the proposed residual layout (ablation) | Ours · Static NeRF, 8 views | Avg. PSNR: 24.74 -> 18.77 | Hurt |
| remove skip connections (ablation) | Ours · Static NeRF, 8 views | Avg. PSNR: 24.74 -> 19.23 | Hurt |
| residual-concatenate coordinates only, rather than coordinates and multi-plane features (ablation) | Ours · Static NeRF, 8 views | Avg. PSNR: 24.74 -> 19.07 | Hurt |
| reduce parameter count to 1.0M (ablation) | Ours · D-NeRF, 25 poses | Avg. PSNR: 25.17 -> 25.10 | No clear effect |
| reduce channel dimension (ablation) | K-Planes · D-NeRF, 25 poses | Avg. PSNR: 23.85 -> 23.41 | Hurt |
| reduce channel dimension (ablation) | HexPlane · D-NeRF, 25 poses | Avg. PSNR: 24.00 -> 22.08 | Hurt |
| add curriculum weighting for multi-plane channels (ablation) | Ours · Static NeRF; drums | Mean PSNR: 19.84 -> 20.20 | Helped |
| add curriculum weighting for multi-plane channels (ablation) | Ours · D-NeRF; standup | Mean PSNR: 25.59 -> 26.22 | Helped |
| add curriculum weighting for multi-plane channels (ablation) | Ours · D-NeRF; hellwarrior | Mean PSNR: 19.83 -> 19.46 | Hurt |
| vary denoising regularization weight lambda_1 (ablation) | Ours · Static NeRF, 8 views | Avg. PSNR: 23.68; 24.47; 24.55; 24.23; 22.99 | Mixed |
Surprising findings
- Explicit grid/plane baselines (iNGP, K-Planes, CAM) still produce high-frequency spectrum at low resolution, failing to isolate low-frequency content.
- Straightforward residual connections (skip at every layer or no skip) lead to ineffective training under sparse inputs, with PSNR around 19 versus 24.74.
Limitations
- Regularization hyperparameters (lambda_1, curriculum schedule, initial resolution) vary per scene and require tuning.
- Rendering is slower than baselines, and the method may underfit scenes needing intricate detail (per appendix).
- Stated failure cases for total variation/Laplacian regularization, which the method sidesteps but does not fully resolve.
Experimental setup
Datasets: NeRF-synthetic (Blender) D-NeRF Tanks and Temples
Benchmarks: NeRF-synthetic D-NeRF Tanks and Temples 2D image regression
Metrics: PSNR SSIM LPIPS variance of PSNR training time rendering time parameter count
Baselines: Simplified NeRF DietNeRF HALO FreeNeRF DVGO VGOS iNGP TensoRF K-Planes HexPlane
Builds on: TensoRF HexPlane NeRF FreeNeRF
Closest papers by meaning
- Mask-Based Modeling for Neural Radiance Fields (ICLR 2024 · similarity 0.87)
- Is Vanilla MLP in Neural Radiance Field Enough for Few-shot View Synthesis? (CVPR 2024 · similarity 0.86)
- Learning Robust Generalizable Radiance Field with Visibility and Feature Augmented Point Representation (ICLR 2024 · similarity 0.86)
- Improving Robustness for Joint Optimization of Camera Pose and Decomposed Low-Rank Tensorial Radiance Fields (AAAI 2024 · similarity 0.86)
- ResFields: Residual Neural Fields for Spatiotemporal Signals (ICLR 2024 · similarity 0.86)
- ColNeRF: Collaboration for Generalizable Sparse Input Neural Radiance Field (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.