Research map / 3D vision / Depth and stereo
Plenodium: Underwater 3D Scene Reconstruction with Plenoptic Medium Representation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Plenodium couples a 3D Gaussian splatting renderer with a plenoptic medium model (position- and direction-dependent spherical harmonics on a voxel grid), plus pseudo-depth Gaussian initialization and a depth ranking loss, for underwater 3D reconstruction and restoration.
Problem
Underwater 3D scene reconstruction is degraded by scattering media: existing medium models depend only on view direction, MLP medium fields are slow, and COLMAP initialization fails in degraded underwater images.
Approach
Model medium color and attenuation/backscatter coefficients with explicit SH coefficients stored on a 3D grid and trilinearly interpolated (encoding both direction and position), initialize Gaussians by complementing COLMAP points with affine-calibrated Depth Anything pseudo-depth, and add a downsampled depth-ordinal ranking loss.
Key results
- On the SeaThru-NeRF dataset, Plenodium averages higher PSNR/SSIM than WaterSplatting (e.g., 34.120 vs 32.203 PSNR on Curaçao) and renders at 249 FPS vs 169 FPS for WaterSplatting, with ~7.0 min training.
- The paper reports a 0.872 dB average PSNR gain and 47% rendering speedup over WaterSplatting on real scenes.
- On the simulated dataset, Plenodium achieves the best PSNR on most hard-degradation subsets, e.g., 25.275 vs 22.510 (WaterSplatting) for fog-hard street scenes.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Plenodium rather than WaterSplatting | underwater scene reconstruction · SeaThru-NeRF dataset | PSNR: +0.872dB on average | Helped |
| Use Plenodium rather than WaterSplatting | underwater scene reconstruction · SeaThru-NeRF dataset | FPS: from 169FPS to 249FPS; 47% | Helped |
| Replace directional MLP medium representation with SH (ablation) | WaterSplatting · SeaThru-NeRF dataset | PSNR: 29.600 -> 30.124; 0.524 dB | Helped |
| Replace directional MLP medium representation with SH (ablation) | WaterSplatting · SeaThru-NeRF dataset | FPS: 179 -> 265; 86 FPS | Helped |
| Add positional encoding via trilinear interpolation (ablation) | SH medium representation without direction · SeaThru-NeRF dataset | PSNR: 29.503 -> 29.796 | Helped |
| Add directional information to positional SH encoding (ablation) | SH medium representation · SeaThru-NeRF dataset | PSNR: 29.796 -> 30.254 | Helped |
| Add PDGC to COLMAP initialization (ablation) | Plenodium · IUI3 Red Sea | PSNR: 30.176 -> 30.275; 0.099dB | Helped |
| Add PDGC to COLMAP initialization (ablation) | Plenodium · Curaçao | PSNR: 33.996 -> 34.120; 0.124dB | Helped |
| Add PDGC to COLMAP initialization (ablation) | Plenodium · Japanese Garden Red Sea | PSNR: 24.947 -> 25.058; 0.111dB | Helped |
| Add PDGC to COLMAP initialization (ablation) | Plenodium · Panama | PSNR: 32.434 -> 32.435; 0.001dB | Helped |
| Replace random Gaussian placement with PDGC (ablation) | COLMAP initialization · SeaThru-NeRF dataset | PSNR: 30.236 -> 30.472 | Helped |
| Add depth ranking loss to L1 and single-scale SSIM (ablation) | Plenodium loss · SeaThru-NeRF dataset | PSNR: 30.307 -> 30.389 | Helped |
Surprising findings
- Replacing the medium MLP with SH alone (SH w/ dir) raises PSNR while reducing time, and position-only SH without direction still improves over the direction-only baseline in the ablation.
- PDGC gives only 0.001 dB PSNR gain on the Panama scene, indicating limited benefit in some scenes.
Limitations
- Relies on a monocular depth model (Depth Anything) whose affine-calibrated pseudo-depth quality bounds initialization gains.
- Gains on some real scenes are marginal (e.g., Panama PDGC +0.001 dB; LPIPS is worse than SeaThru-NeRF on several scenes).
Experimental setup
Datasets: SeaThru-NeRF dataset simulated Blender underwater dataset
Benchmarks: SeaThru-NeRF dataset Plenodium simulated dataset
Metrics: PSNR SSIM LPIPS FPS training time
Baselines: Zip-NeRF SeaThru-NeRF 3D Gaussian splatting WaterSplatting
Builds on: 3D Gaussian splatting WaterSplatting SeaThru-NeRF Depth Anything COLMAP
Closest papers by meaning
- Atlantis: Enabling Underwater Depth Estimation with Stable Diffusion (CVPR 2024 · similarity 0.86)
- RestorGS: Depth-aware Gaussian Splatting for Efficient 3D Scene Restoration (CVPR 2025 · similarity 0.85)
- Neural Underwater Scene Representation (CVPR 2024 · similarity 0.85)
- NAUTILUS: A Large Multimodal Model for Underwater Scene Understanding (NeurIPS 2025 · similarity 0.84)
- WaterDiffusion: Learning a Prior-involved Unrolling Diffusion for Joint Underwater Saliency Detection and Visual Restoration (AAAI 2025 · similarity 0.84)
- Decoupling Scattering: Pseudo-Label Guided NeRF for Scenes with Scattering Media (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.