Research map / 3D vision / NeRF and radiance fields
NeRFCodec: Neural Feature Compression Meets Neural Radiance Fields for Memory-Efficient Scene Representation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes NeRFCodec, an end-to-end NeRF compression framework that reuses a pretrained neural 2D image codec (Cheng et al.) with content-specific encoder and decoder head, trained with rendering and entropy losses to compress TensoRF feature planes to about 0.5 MB per scene.
Problem
Storage-efficient representation of hybrid NeRF scenes: explicit feature grids still require tens to hundreds of megabytes, and training neural transform codecs from scratch on large collections of feature planes is infeasible.
Approach
Reuse a pretrained neural image codec by replacing its first encoder layer and last decoder layer, fine-tune these content-specific parts per scene under rendering plus entropy loss, and freeze the remaining decoder backbone, enabling non-linear transform coding of NeRF feature planes.
Key results
- Achieves 0.453 MB total size with 35.08 dB PSNR on the Chair scene ablation setting, versus 1.786 MB for a non-tuned image codec baseline.
- Memory breakdown shows feature planes compressed from 69.953 MB to 0.052 MB (about 1345x), with total 0.453 MB from 72.342 MB (about 160x).
- Reports rate-distortion frontier improvements over VQRF, Masked Wavelet NeRF and BiRF across NeRF-Synthetic, NSVF-Synthetic and Tanks and Temples, with a single scene at 0.5 MB.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| remove the feature encoder and learn the latent code directly (ablation) | NeRFCodec · Chair, NeRF-Synthetic | PSNR: 35.08 -> 34.41 dB | Hurt |
| remove the feature encoder and learn the latent code directly (ablation) | NeRFCodec · Chair, NeRF-Synthetic | size: 0.453 -> 0.461 MB | Hurt |
| reuse and jointly fine-tune an unmodified 2D image codec (ablation) | NeRFCodec · Chair, NeRF-Synthetic | PSNR: 33.25 -> 35.08 dB | Hurt |
| reuse and jointly fine-tune an unmodified 2D image codec (ablation) | NeRFCodec · Chair, NeRF-Synthetic | size: 0.453 -> 1.150 MB | Hurt |
| remove entropy loss (ablation) | NeRFCodec · Chair, NeRF-Synthetic | size: 0.453 -> 0.701 MB | Hurt |
| remove entropy loss (ablation) | NeRFCodec · Chair, NeRF-Synthetic | PSNR: 35.08 -> 34.98 dB | No clear effect |
| add feature-plane reconstruction loss (ablation) | NeRFCodec · Chair, NeRF-Synthetic | PSNR: 35.08 -> 32.81 dB | Hurt |
| add feature-plane reconstruction loss (ablation) | NeRFCodec · Chair, NeRF-Synthetic | SSIM: 0.981 -> 0.965 | Hurt |
| adapt the codec on eight NeRF-Synthetic scenes, then apply it to unseen NSVF-Synthetic scenes without test-time optimization (ablation) | NeRFCodec feature codec · NSVF-Synthetic | rendering quality | Hurt |
Surprising findings
- Naively reusing a pretrained 2D image codec on feature planes without modification gives much higher bitrate (1.786 MB) than the content-adaptive approach.
- Adding a feature reconstruction loss against the pretrained feature planes degrades rendering quality (32.81 vs 35.08 dB).
Limitations
- Training the non-linear transform is time-consuming.
- A specialized neural feature codec must be trained for each scene individually.
- A codec trained on a few NeRF-Synthetic objects fails to generalize to NSVF-Synthetic feature planes.
Experimental setup
Datasets: NeRF-Synthetic NSVF-Synthetic Tanks and Temples
Benchmarks: NeRF-Synthetic NSVF-Synthetic Tanks and Temples
Metrics: PSNR SSIM memory size (MB)
Baselines: VQRF Masked Wavelet NeRF BiRF VQ-TensoRF TensoRF Instant-NGP K-Planes DVGO Plenoxels
Builds on: TensoRF Cheng et al. neural image codec CompressAI Balle hyperprior Masked Wavelet NeRF
Closest papers by meaning
- Neural NeRF Compression (ICML 2024 · similarity 0.95)
- CodecNeRF: Toward Fast Encoding and Decoding, Compact, and High-quality Novel-view Synthesis (AAAI 2025 · similarity 0.94)
- ACRF: Compressing Explicit Neural Radiance Fields via Attribute Compression (ICLR 2024 · similarity 0.93)
- How Far Can We Compress Instant-NGP-Based NeRF? (CVPR 2024 · similarity 0.89)
- NeRF-VPT: Learning Novel View Representations with Neural Radiance Fields via View Prompt Tuning (AAAI 2024 · similarity 0.89)
- Mask-Based Modeling for Neural Radiance Fields (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.