Research map / 3D vision / Face and shape
Looking Into the Water by Unsupervised Learning of the Surface Shape
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Unsupervised restoration of underwater scenes seen through a wavy water surface, using two SIREN neural fields (surface height over space-time and a latent undistorted image) trained by reconstructing the observed distorted frames.
Problem
Removing refraction-induced distortions from air-captured images of underwater scenes caused by a dynamic water surface, without reliable real-world ground truth for supervised training.
Approach
Model the water surface height with a time-conditioned SIREN and derive per-pixel refraction offsets from its spatial gradients via Snell's law; a second SIREN models the static underlying image. Training uses only a single L1 reconstruction loss over the observed frame sequence, which also yields a surface height estimate.
Key results
- On Real1 the method averages PSNR 19.78, SSIM 0.61, LPIPS 0.12, versus NDIR 19.44 / 0.59 / 0.14.
- On the synthetic dataset it beats NDIR on all three wave types, e.g., Ripple PSNR 23.07 vs 21.50 and LPIPS 0.05 vs 0.07.
- Surface height on synthetic data reaches RMSE 0.115 and Abs Rel 0.0635.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use L1 reconstruction loss only instead of L2 loss only (ablation) | Ours · Real1 average | PSNR: 18.79 -> 19.78 | Helped |
| use L1 reconstruction loss only instead of L3 loss only (ablation) | Ours · Real1 average | PSNR: 15.32 -> 19.78 | Helped |
| remove the added L2 and L3 loss terms, retaining L1 only (ablation) | Ours · Real1 average | LPIPS: 0.13 -> 0.121 | Helped |
| add the initialization training stage (ablation) | Ours · Real1 average | LPIPS: 0.19 -> 0.121 | Helped |
| add random Fourier positional encoding (ablation) | Ours · Real1 average | LPIPS: 0.25 -> 0.121 | Helped |
| predict per-frame distortions with separate networks instead of a time-conditioned surface-height network (ablation) | Ours · Real1 average | PSNR: 19.42 -> 19.78 | Helped |
| model shared per-frame distortions without estimating surface height (ablation) | Ours · Real1 average | PSNR: 19.47 -> 19.78 | Helped |
| increase input sequence size from 5 to 10 frames (ablation) | Ours · Real1 average | PSNR: 17.69 -> 19.78 | Helped |
| use Ours rather than NDIR | Image restoration · Synthetic Gaussian, Ripple, and Ocean | PSNR: 20.70 -> 20.92; 21.50 -> 23.07; 15.52 -> 16.07 | Helped |
| use Ours rather than NDIR | Image restoration · Real1 average | LPIPS: 0.14 -> 0.12 | Helped |
Surprising findings
- NDIR achieves higher PSNR than the proposed method on the Math sequence and at batch size 5, and NeRT gets strong PSNR/SSIM but much worse LPIPS.
- Reducing the NDIR training loss from three terms to a single reconstruction term worked better for this model.
Limitations
- Assumes orthographic camera, fronto-parallel planar scene, and small waves relative to average water height.
- Motion blur and non-planar scenes violate assumptions; some detail loss from limited network resolution.
Experimental setup
Datasets: James Real1 dataset TianSet synthetic wave dataset following prior simulation method
Benchmarks: James Real1 dataset TianSet synthetic wave dataset
Metrics: PSNR SSIM LPIPS RMSE Abs Rel
Baselines: NDIR Li et al. unsupervised/supervised method NeRT
Builds on: SIREN NDIR random Fourier features
Closest papers by meaning
- Neural Underwater Scene Representation (CVPR 2024 · similarity 0.83)
- ResFields: Residual Neural Fields for Spatiotemporal Signals (ICLR 2024 · similarity 0.83)
- Decoupling Scattering: Pseudo-Label Guided NeRF for Scenes with Scattering Media (AAAI 2025 · similarity 0.82)
- Auto-Linear Phenomenon in Subsurface Imaging (ICML 2024 · similarity 0.82)
- NeuralIndicator: Implicit Surface Reconstruction from Neural Indicator Priors (ICML 2024 · similarity 0.82)
- Continuous Field Reconstruction from Sparse Observations with Implicit Neural Networks (ICLR 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.