Research map / 3D vision / NeRF and radiance fields
Coordinate-Aware Modulation for Neural Fields
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Coordinate-Aware Modulation (CAM), which uses grid-parameterized scale and shift values (from interpolated single-channel grids) to modulate MLP intermediate features in neural fields, combined with feature normalization, and reports gains across image, NeRF, dynamic NeRF, and video tasks.
Problem
Neural fields built on MLPs suffer from spectral bias and slow convergence, while grid-based methods are fast and bias-free but memory-heavy; prior methods combine them sequentially, leaving the MLP in a secondary role.
Approach
Instead of feeding grid features into an MLP, CAM parallelly modulates intermediate MLP features with per-channel-agnostic scalar scale and shift values interpolated from small single-channel grids indexed by a chosen subset of coordinates (prioritizing time or view direction), and normalizes features before modulation.
Key results
- On D-NeRF dynamic scenes, NerfAcc+CAM reaches 33.78 PSNR with 0.6M params vs 32.22 for NerfAcc and 32.67 for TiNeuVox (12M params).
- On the 360 real-scene dataset, Mip-NeRF 360+CAM achieves 29.98 PSNR vs 29.11 for Mip-NeRF 360 (9M params).
- On NeRF synthetic at 8-bit, Mip-NeRF+CAM reaches 33.27 PSNR at 0.58 MB vs 32.86 for Mip-NeRF at the same size.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add CAM to FFN | FFN · Natural and Text image regression and generalization | PSNR: 30.30 -> 32.21; 34.44 -> 50.17; 27.48 -> 28.19; 30.04 -> 33.09 | Helped |
| Add CAM to Mip-NeRF | Mip-NeRF · NeRF Synthetic, NSVF Synthetic, and LLFF, 32-bit | PSNR: 33.09 -> 33.42; 35.83 -> 36.56; 26.86 -> 27.17 | Helped |
| Add CAM to Mip-NeRF 360 | Mip-NeRF 360 · 360 dataset | PSNR: 29.11 -> 29.98 | Helped |
| Add CAM to NerfAcc | NerfAcc · D-NeRF | PSNR: 32.22 -> 33.78 | Helped |
| Add CAM to FFNeRV | FFNeRV · UVG video at 32-, 8-, and 6-bit | PSNR: 35.70 -> 35.95; 35.55 -> 35.86; 34.85 -> 35.45 | Helped |
| Use time coordinates rather than space or direction coordinates for CAM (ablation) | NerfAcc on D-NeRF · D-NeRF coordinate priority | PSNR: 32.22 -> 33.78 | Helped |
| Use space coordinates for CAM (ablation) | NerfAcc on D-NeRF · D-NeRF coordinate priority | PSNR: 32.22 -> 32.57 | Helped |
| Use direction coordinates for CAM (ablation) | NerfAcc on D-NeRF · D-NeRF coordinate priority | PSNR: 32.22 -> 32.44 | Helped |
| Use both direction and time coordinates for CAM (ablation) | NerfAcc on D-NeRF · D-NeRF coordinate priority | PSNR: 32.22 -> 32.49 | Helped |
| Use space coordinates for CAM (ablation) | Mip-NeRF · Mip-NeRF coordinate priority | PSNR: 33.09 -> 32.70 | Hurt |
| Use direction coordinates for CAM (ablation) | Mip-NeRF · Mip-NeRF coordinate priority | PSNR: 33.09 -> 33.42 | Helped |
| Add feature normalization to CAM (ablation) | CAM on Natural images · Image, Natural | PSNR: 30.9 -> 32.2 | Helped |
Surprising findings
- Normalization (BN, LN, IN) applied without CAM typically degrades performance, while normalization combined with CAM improves it.
- Modulating with a single prioritized coordinate (time for dynamic NeRF) outperforms using both time and view direction together, and spatial-coordinate modulation for NeRF yields a 13.1M-param model with lower gains than the 0.6M variant.
Limitations
- Normalization over large units (e.g., full 4D video tensors) increases training time by about 50%, forcing a compromise design for video.
- Spatial-coordinate modulation becomes costly for high-dimensional inputs due to grid curse of dimensionality, requiring a coordinate priority heuristic.
- CAM can slow inference at small test chunk sizes (e.g., 0.51 to 0.26 FPS for NerfAcc vs NerfAcc+CAM at chunk 1024).
Experimental setup
Datasets: Natural image dataset Text image dataset NeRF synthetic NSVF synthetic LLFF Mip-NeRF 360 real scenes D-NeRF UVG
Benchmarks: NeRF synthetic NSVF synthetic LLFF 360 dataset D-NeRF UVG
Metrics: PSNR parameter count model size (MB) training time BPP FPS
Baselines: NeRF TensoRF Mip-NeRF Mip-NeRF 360 NerfAcc K-planes D-NeRF TiNeuVox I-NGP Zip-NeRF FFNeRV NeRV
Builds on: Mip-NeRF NerfAcc FFNeRV FFN I-NGP
Closest papers by meaning
- ResFields: Residual Neural Fields for Spatiotemporal Signals (ICLR 2024 · similarity 0.86)
- FreSh: Frequency Shifting for Accelerated Neural Representation Learning (ICLR 2025 · similarity 0.85)
- Neural Fields as Distributions: Signal Processing Beyond Euclidean Space (CVPR 2024 · similarity 0.85)
- Grids Often Outperform Implicit Neural Representation at Compressing Dense Signals (NeurIPS 2025 · similarity 0.85)
- Synergistic Integration of Coordinate Network and Tensorial Feature for Improving Neural Radiance Fields from Sparse Inputs (ICML 2024 · similarity 0.85)
- How to Train Neural Field Representations: A Comprehensive Study and Benchmark (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.