Research map / 3D vision / Depth and stereo
Single View Refractive Index Tomography with Neural Fields
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Reconstructs a 3D continuous refractive index field from a single view by differentiably ray-tracing curved rays through a coordinate MLP, using known light sources inside the volume as supervision.
Problem
Single-view refractive index tomography: recovering a continuously varying 3D refractive index field from one 2D image, where ray paths depend nonlinearly on the unknown field and the problem is severely ill-posed.
Approach
Model the refractive field as a coordinate-based MLP with softplus output and optimize it through differentiable Hamiltonian ray tracing (adjoint method) so that rendered images of known light sources inside the medium match observations; light sources distributed throughout the volume make single-view recovery feasible.
Key results
- Neural field reconstruction from single view with light sources inside the volume reaches 30.7 dB PSNR on gas flow, versus 25.8 dB and 25.6 dB for voxel grid variants with and without TV2 under other conditions.
- With 290 light sources, smooth random refractive fields are recovered with visible structure, though the method is susceptible to artifacts in some cases.
- Recovery of the dark matter refractive field from a simulated single telescope image correlates strongly with true halos despite a median deflection of only 6e-3 pixels.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a neural field instead of an unregularized voxel grid | Single-view RI tomography · Fuel injection dataset, light sources throughout volume | PSNR: 20.2 dB -> 30.7 dB | Helped |
| Use a neural field instead of a voxel grid with TV2 regularization | Single-view RI tomography · Fuel injection dataset, light sources throughout volume | PSNR: 25.3 dB -> 30.7 dB | Helped |
| Use light sources throughout the volume instead of behind it | Neural field reconstruction · Fuel injection dataset | PSNR: 24.2 dB -> 30.7 dB | Helped |
| Increase the number of light sources from 50 to 250 | Neural field reconstruction · Refractive field with varying light source density | PSNR: 22.2 dB -> 25.3 dB | Helped |
Surprising findings
- Ray displacement measurements from a plane of light sources behind the volume fail to recover the gas flow shape from a single view, even with perfect optical flow.
- A neural field with implicit regularization outperforms a voxel grid even when the voxel grid uses TV2 regularization tuned on ground truth.
Limitations
- Sensitive to noise in light source locations and image measurements (unmodeled noise left to future work).
- Highly non-convex optimization can yield reconstruction artifacts; sparse light sources degrade background recovery.
- Dark matter experiment uses noiseless simulation with Gaussian emitters at halo centers.
Experimental setup
Datasets: fuel injection gas flow dataset (SFB 382, 64^3 RI grid) IllustrisTNG TNG300-1 dark matter halo catalog procedurally generated Gaussian process refractive fields
Benchmarks: simulated refractive fields (gas flow, smooth GP fields) simulated dark matter halo recovery (IllustrisTNG)
Metrics: PSNR of refractive index (RI minus 1) image fit (MSE)
Baselines: linear voxel grid voxel grid with TV2 regularization
Builds on: NeRF Hamiltonian ray tracing Dormand-Prince integrator Background-oriented Schlieren Eikonal neural field refractive tomography
Closest papers by meaning
- Revealing the 3D Cosmic Web through Gravitationally Constrained Neural Fields (ICLR 2025 · similarity 0.86)
- OpticalNet: An Optical Imaging Dataset and Benchmark Beyond the Diffraction Limit (CVPR 2025 · similarity 0.84)
- Inverse Rendering of Glossy Objects via the Neural Plenoptic Function and Radiance Fields (CVPR 2024 · similarity 0.83)
- Differentiable Neural Surface Refinement for Modeling Transparent Objects (CVPR 2024 · similarity 0.82)
- Light Transport-aware Diffusion Posterior Sampling for Single-View Reconstruction of 3D Volumes (CVPR 2025 · similarity 0.82)
- Decoupling Scattering: Pseudo-Label Guided NeRF for Scenes with Scattering Media (AAAI 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.