Research map / 3D vision / NeRF and radiance fields
IReNe: Instant Recoloring of Neural Radiance Fields
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes IReNe, which recolors a pre-trained Instant-NGP NeRF in about 5 seconds from a single user-edited image by retraining only diffuse neurons of the last color-MLP layer and using a learned soft 3D segmentation field.
Problem
Editing the color of NeRFs interactively while keeping photorealism, object boundary precision, and multi-view consistency; existing methods are slow (minutes to hours) and suffer color bleeding and view inconsistency.
Approach
Retrain only the last layer of the color MLP, freeze neurons identified automatically as view-dependent (via angular activation profiles), and blend new and original colors with a soft 3D segmentation MLP operating on hash-grid features, supervised only by the single edited image.
Key results
- IReNe achieves PSNR 30.33 on NeRF Synthetic versus 29.92 for PaletteNeRF and 29.00 for RecolorNeRF, and the best LPIPS on that set (0.035).
- IReNe trains in about 5 seconds per scene, compared with 25 s for ICE-NeRF, 15-20 min for RecolorNeRF, and 30 min to 2 h for PaletteNeRF, reported as 5x to 500x speedups.
- Ablation: last-layer-only training beats full MLP training (NeRF Synthetic PSNR 29.09 vs 28.25), and adding soft segmentation and view-dependent neuron selection yields the best results (IReNe 30.33).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train only the last color MLP layer instead of the full color MLP (ablation) | IReNe · NeRF Synthetic, LLFF, and Mip NeRF 360 | PSNR: 28.25 -> 29.09; 24.86 -> 24.99; 24.31 -> 25.35 | Helped |
| Add soft segmentation to last-layer training (ablation) | IReNe · NeRF Synthetic and Mip NeRF 360 | PSNR: 29.09 -> 30.20; 25.35 -> 25.68 | Helped |
| Add soft segmentation to last-layer training (ablation) | IReNe · LLFF | PSNR: 24.99 -> 24.77 | Hurt |
| Add diffuse-neuron selection, preserving view-dependent neurons (ablation) | IReNe with soft segmentation · NeRF Synthetic, LLFF, and Mip NeRF 360 | PSNR: 30.20 -> 30.33; 24.77 -> 25.63; 25.68 -> 26.80 | Helped |
| Use IReNe rather than PaletteNeRF | NeRF Synthetic, LLFF, and Mip NeRF 360 · NeRF Synthetic, LLFF, and Mip NeRF 360 | PSNR: 29.92 -> 30.33; 24.02 -> 25.63; 22.24 -> 26.80 | Helped |
| Use IReNe rather than RecolorNeRF | NeRF Synthetic and LLFF · NeRF Synthetic and LLFF | PSNR: 29.00 -> 30.33; 24.07 -> 25.63 | Helped |
Surprising findings
- Retraining only the last layer of the color MLP gives better quality than retraining the full MLP.
Limitations
- Relies on external tools such as Photoshop to complete the edit.
- Soft segmentation results may be poor on some occasions.
- Does not model indirect illumination effects of edited objects on other objects.
Experimental setup
Datasets: NeRF Synthetic LLFF Mip-NeRF 360
Benchmarks: NeRF Synthetic LLFF Mip-NeRF 360
Metrics: PSNR SSIM LPIPS training time
Baselines: PaletteNeRF RecolorNeRF ICE-NeRF Instant-NGP
Builds on: Instant-NGP PaletteNeRF ICE-NeRF RecolorNeRF
Closest papers by meaning
- Bright-NeRF: Brightening Neural Radiance Field with Color Restoration from Low-Light RAW Images (AAAI 2025 · similarity 0.90)
- LAENeRF: Local Appearance Editing for Neural Radiance Fields (CVPR 2024 · similarity 0.89)
- NeRF On-the-go: Exploiting Uncertainty for Distractor-free NeRFs in the Wild (CVPR 2024 · similarity 0.89)
- InsertNeRF: Instilling Generalizability into NeRF with HyperNet Modules (ICLR 2024 · similarity 0.88)
- Rad-NeRF: Ray-decoupled Training of Neural Radiance Field (NeurIPS 2024 · similarity 0.88)
- Language-driven Object Fusion into Neural Radiance Fields with Pose-Conditioned Dataset Updates (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.