Research map / 3D vision / 3D generation
See through the Dark: Learning Illumination-affined Representations for Nighttime Occupancy Prediction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LIAR, a nighttime vision-based occupancy prediction framework that selectively enhances low-light images via Retinex illumination maps and uses illumination-guided 2D sampling and 3D illumination-driven projection to handle under- and overexposure.
Problem
Vision-based 3D occupancy prediction degrades sharply at night due to limited visibility, underexposed regions that degrade features, and overexposed regions (headlights, streetlamps) that cause semantic deficiency.
Approach
Estimate a Retinex illumination map, selectively enhance only genuinely low-light images using an Otsu-style threshold, modulate 2D deformable sampling offsets by darkness (larger offsets in dark regions), and build a 3D illumination intensity field by projecting BEV grid heights into the illumination map to reweight residual BEV queries.
Key results
- On Occ3D-nuScenes nighttime-only training, LIAR (1f) reaches 19.27 mIoU, 5.85 above the second-best method.
- LIAR (2f) reaches 22.09 mIoU on night test split versus 20.01 for COTR (2f) when trained on night data only.
- On nuScenes-C hard severity, LIAR (1f) outperforms BEVFormer (4f) by 6.50 mIoU; ablation shows SLLIE adds 0.97 mIoU over baseline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use LIAR instead of FlashOcc (1f) | Occupancy prediction · Occ3D-nuScenes night train, night test (1f) | mIoU | Helped |
| use LIAR instead of COTR (2f) | Occupancy prediction · Occ3D-nuScenes day and night train, night test (2f) | mIoU: 2.08 | Helped |
| use LIAR instead of the strongest listed baseline | Occupancy prediction · nuScenes-C Dark, across easy, moderate, and hard severities | mIoU | Helped |
| add SLLIE to the baseline (ablation) | LIAR · Occ3D-nuScenes night split | mIoU: +0.97 mIoU | Helped |
| add illumination by direct addition (ablation) | LIAR · Occ3D-nuScenes night split | mIoU: +0.44 mIoU | Helped |
| add illumination by concatenation (ablation) | LIAR · Occ3D-nuScenes night split | mIoU: +0.39 mIoU | Helped |
| use 2D-IGS instead of naive illumination fusion (ablation) | LIAR · Occ3D-nuScenes night split | mIoU | Helped |
| add 3D-IDP (ablation) | LIAR · Occ3D-nuScenes night split | mIoU: +1.46 mIoU | Helped |
| add SLLIE, 2D-IGS, and 3D-IDP (ablation) | LIAR · Occ3D-nuScenes night split | mIoU: +1.89 mIoU | Helped |
| enhance every nighttime image indiscriminately (threshold t = 1.00) (ablation) | SLLIE · Occ3D-nuScenes night split | mIoU: 14.39 to 14.08 | Hurt |
| replace illumination-guided sampling with standard DCN (ablation) | 2D-IGS · Occ3D-nuScenes night split | mIoU: 14.11 to 13.81 | Hurt |
| remove illumination guidance (ablation) | 2D-IGS · Occ3D-nuScenes night split | mIoU: -0.24 mIoU | Hurt |
Surprising findings
- Indiscriminately enhancing all nighttime images (threshold 1.00) lowers mIoU from 14.39 to 14.08, so enhancement can hurt well-lit images.
- Naive fusion of illumination maps with image features yields only marginal gains (0.44 and 0.39 mIoU).
Limitations
- Nighttime subset of Occ3D-nuScenes is small and some categories (bus, construction vehicle, trailer) are absent.
- nuScenes-C is synthetic and differs from real nighttime driving (no sensor degradations like thermal noise or motion blur).
- Added computational cost from the Retinex decomposition network reduces FPS.
Experimental setup
Datasets: Occ3D-nuScenes nuScenes-C nuScenes
Benchmarks: Occ3D-nuScenes nuScenes-C
Metrics: mIoU
Baselines: BEVDetOcc FlashOcc SparseOcc OPUS BEVFormer FBOcc COTR LCDPNet CSEC
Builds on: FlashOcc SCI BEVPooling deformable convolution MonoScene
Closest papers by meaning
- NightCC: Nighttime Color Constancy via Adaptive Channel Masking (CVPR 2024 · similarity 0.87)
- Exploring the Common Appearance-Boundary Adaptation for Nighttime Optical Flow (ICLR 2024 · similarity 0.86)
- NightHaze: Nighttime Image Dehazing via Self-Prior Learning (AAAI 2025 · similarity 0.86)
- LowRankOcc: Tensor Decomposition and Low-Rank Recovery for Vision-based 3D Semantic Occupancy Prediction (CVPR 2024 · similarity 0.85)
- The Parables of the Mustard Seed and the Yeast: Extremely Low-Budget, High-Performance Nighttime Semantic Segmentation (AAAI 2025 · similarity 0.85)
- Seeing in the Dark: Benchmarking Egocentric 3D Vision with the Oxford Day-and-Night Dataset (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.