Research map / 3D vision / NeRF and radiance fields
Colorizing Monochromatic Radiance Fields
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ColorNeRF, which colorizes monochromatic radiance fields by predicting missing Lab ab-channel representations from a frozen image colorizer (CT^2) via query-based colorization, histogram-guided purification, and classification-based color injection.
Problem
NeRF cannot produce colorful 3D representations when only monochromatic images are available; existing colorize-then-fuse and style/color-editing NeRF methods yield implausible (view-inconsistent) or unvivid colors.
Approach
Treat NeRF colorization as representation prediction in CIE Lab: first fit luminance and density fields from monochrome images, then predict ab color representation by querying a frozen colorization model on rendered luminance patches, purifying outliers via histogram similarity, and supervising a 313-bin ab classification head to keep vivid colors.
Key results
- ColorNeRF achieves PSNR 20.76, SSIM 0.81, LPIPS 0.21, Colorful 55.40 and Delta Colorful 12.15 on synthetic monochromatic data, best on all metrics versus CT^2+NeRF (PSNR 18.90, Colorful 48.52).
- User study (50 participants, 1-5 scale) gives ColorNeRF plausibility 4.26 and vividness 4.67, versus 3.25 and 3.40 for CT^2+NeRF.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add histogram-guided purification (ablation) | ColorNeRF · Synthetic monochromatic data | Δ Colorful: 19.20 -> 12.15 | Helped |
| Add histogram-guided purification (ablation) | ColorNeRF · Synthetic monochromatic data | Colorful Score: 54.41 -> 55.40 | Hurt |
| Add classification-based color injection (ablation) | ColorNeRF · Synthetic monochromatic data | Colorful Score: 49.28 -> 55.40 | Helped |
| Add classification-based color injection (ablation) | ColorNeRF · Synthetic monochromatic data | PSNR: 19.22 -> 20.76 | Helped |
| Use ColorNeRF instead of CT²+NeRF | ColorNeRF · Synthetic monochromatic data | LPIPS: 0.32 -> 0.21 | Helped |
Surprising findings
- Querying the colorizer repeatedly with varying sampled patches and averaging over the resulting colors yields more consistent results than fixing one colorization per image.
Limitations
- Performance depends on the external colorization module; L-CoDer yields artifacts with uncertain background colors.
- Pixel metrics like PSNR and SSIM may not reflect colorization quality, so reliance on a user study.
Experimental setup
Datasets: LLFF self-captured monochromatic scenes spike camera recordings old movie clips
Benchmarks: LLFF synthetic monochromatic data (flower, trex, 8 captured scenes)
Metrics: PSNR SSIM LPIPS Colorful score Delta Colorful user study plausibility user study vividness
Baselines: ARF CLIP-NeRF CT^2+NeRF Vid+NeRF
Builds on: NeRF GRAF CT^2 COLMAP
Closest papers by meaning
- Bright-NeRF: Brightening Neural Radiance Field with Color Restoration from Low-Light RAW Images (AAAI 2025 · similarity 0.89)
- SpectralNeRF: Physically Based Spectral Rendering with Neural Radiance Field (AAAI 2024 · similarity 0.85)
- Learning Robust Generalizable Radiance Field with Visibility and Feature Augmented Point Representation (ICLR 2024 · similarity 0.84)
- A Pre-convolved Representation for Plug-and-Play Neural Illumination Fields (AAAI 2024 · similarity 0.84)
- Aleth-NeRF: Illumination Adaptive NeRF with Concealing Field Assumption (AAAI 2024 · similarity 0.84)
- Mask-Based Modeling for Neural Radiance Fields (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.