Research map / 3D vision / NeRF and radiance fields
Neural NeRF Compression
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Compresses per-scene TensoRF-VM feature planes with an encoder-free, end-to-end optimized neural transform coder that uses importance-weighted rate-distortion loss and a learned masked entropy model.
Problem
Grid-based NeRF models (e.g., TensoRF) render fast but require large storage for their feature grids, hindering practical deployment and transmission.
Approach
Replaces the encoder of conventional neural transform coding with directly optimized per-scene latent codes and a lightweight transposed-conv decoder, weights the feature-reconstruction loss by log rendering-importance maps, and uses a spike-and-slab masked factorized entropy model trained with straight-through Gumbel-Softmax to exploit spatial sparsity.
Key results
- On Synthetic-NeRF, TC-TensoRF-L reaches 32.93 dB PSNR at 3.4 MB versus VQ-TensoRF 32.86 dB at 3.6 MB; BD-rate savings vs VQ-TensoRF are 28.8% on Synthetic-NeRF, 21.1% on Synthetic-NSVF, and 16.7% on Tanks and Temples.
- The high-compression TC-TensoRF-H variant compresses models to about 1.6 MB with 32.31 dB PSNR on Synthetic-NeRF.
- Masked entropy model reduces file size at 32.98 dB from 4.26 MB (factorized prior) to 3.92 MB in medium-to-high rate regimes.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| compress TensoRF-VM with TC-TensoRF-L instead of VQ-TensoRF | TensoRF-VM · Synthetic-NeRF | PSNR: 32.86 -> 32.93 | Helped |
| compress TensoRF-VM with TC-TensoRF-L instead of VQ-TensoRF | TensoRF-VM · Synthetic-NSVF | PSNR: 36.16 -> 36.34 | Helped |
| compress TensoRF-VM with TC-TensoRF-L instead of Re:TensoRF | TensoRF-VM · LLFF | PSNR: 26.55 -> 26.44 | Hurt |
| compress TensoRF-VM with TC-TensoRF-L instead of VQ-TensoRF | TensoRF-VM · Tanks and Temples | PSNR: 28.20 -> 28.42 | Helped |
| use per-scene optimization instead of training NTC across 7 scenes (ablation) | TensoRF-VM compression · Synthetic-NeRF (Lego) | PSNR: 24.52 dB; 12.03 dB lower than uncompressed PSNR (36.55 dB) | Helped |
| remove importance weighting (ablation) | ECTensoRF-L · Synthetic-NeRF at PSNR 32.98 dB | File size (MB): 3.92 MB -> 4.59 MB | Hurt |
| replace the factorized prior with the masked entropy model (ablation) | ECTensoRF-L · Synthetic-NeRF at PSNR 32.98 dB | File size (MB): 4.26 MB -> 3.92 MB | Helped |
| use an encoder rather than directly optimizing latent codes (ablation) | ECTensoRF-L · Synthetic-NeRF | rate-distortion performance | Hurt |
| initialize latents with Gaussian noise instead of zeros (ablation) | ECTensoRF-L · Synthetic-NeRF | PSNR: 32.98 vs. 32.98 | No clear effect |
| train compression model end-to-end from scratch instead of two stages (ablation) | ECTensoRF-L · Synthetic-NeRF at 2e-4 | PSNR: 32.98 -> 31.86 | Hurt |
| add a hyperprior (ablation) | ECTensoRF-L · NeRF-Synthetic at 2e-4 | File size (MB): 3.92 -> 3.72 | Helped |
| add a hyperprior (ablation) | ECTensoRF-L · NeRF-Synthetic at 2e-2 | File size (MB): 1.86 -> 1.92 | Hurt |
Surprising findings
- A transform-coding encoder trained across 7 scenes reached only 24.52 dB on the held-out Lego scene, well below per-scene optimized models.
- End-to-end training from scratch (without a pretrained TensoRF) performed much worse at higher compression, e.g., 25.86 dB at 1.69 MB vs 31.31 dB for the two-stage approach.
Limitations
- Longer compression time than VQ-TensoRF (about 40 vs 7 minutes).
- The decoder must be transmitted, adding fixed storage overhead (about 1.38 MB in the breakdown).
- Slightly behind VQ-TensoRF and Re:TensoRF in PSNR on LLFF; masked model lags factorized prior at low rates.
Experimental setup
Datasets: Synthetic-NeRF Synthetic-NSVF LLFF Tanks and Temples
Benchmarks: Synthetic-NeRF Synthetic-NSVF LLFF Tanks and Temples
Metrics: PSNR SSIM file size (MB) BD-PSNR BD-rate
Baselines: NeRF TensoRF-CP TensoRF-VM VQ-TensoRF Re:TensoRF
Builds on: TensoRF nonlinear transform coding hyperprior Gumbel-Softmax
Closest papers by meaning
- NeRFCodec: Neural Feature Compression Meets Neural Radiance Fields for Memory-Efficient Scene Representation (CVPR 2024 · similarity 0.95)
- CodecNeRF: Toward Fast Encoding and Decoding, Compact, and High-quality Novel-view Synthesis (AAAI 2025 · similarity 0.92)
- How Far Can We Compress Instant-NGP-Based NeRF? (CVPR 2024 · similarity 0.91)
- Rad-NeRF: Ray-decoupled Training of Neural Radiance Field (NeurIPS 2024 · similarity 0.90)
- Mask-Based Modeling for Neural Radiance Fields (ICLR 2024 · similarity 0.89)
- SCINeRF: Neural Radiance Fields from a Snapshot Compressive Image (CVPR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.