Research map / 3D vision / Depth and stereo
Differentiable Neural Surface Refinement for Modeling Transparent Objects
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Refines NeuS surfaces of transparent objects by tracing reflected and refracted rays with Snell's law and a differentiable nested-optimization gradient, trained end-to-end on multi-view RGB images.
Problem
Neural implicit surface reconstruction via volume rendering assumes straight light paths, so reconstructions of transparent objects (refraction and reflection) are poor, and prior refraction-aware methods need controlled setups or known refractive indices.
Approach
Cast reflected and refracted rays with sphere tracing and Snell's law on an initial NeuS surface, render the background color from a pre-learned radiance field along these rays combined via Fresnel terms, and backpropagate the photometric loss to the SDF network using closed-form implicit-differentiation gradients of ray-surface intersections.
Key results
- On six transparent datasets, TNSR achieves the best PSNR/SSIM/LPIPS on Optical Ball (24.07 PSNR), Bottle, Kitty and Ellipsoid (25.24 PSNR on Ellipsoid), outperforming NeuS and NeuS+Ref.
- On synthetic meshes, TNSR lowers chamfer-L1 relative to NeuS on Optical Ball (1.91 vs 1.96), Bottle (29.54 vs 31.05), Kitty (18.05 vs 19.28) and Ellipsoid (0.80 vs 1.05) (x10^-3).
- Refractive index can be searched by maximizing novel-view PSNR, recovering the ground-truth index of 1.45 on Optical Ball without extra training.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add differentiable transparent surface refinement to NeuS | NeuS+Ref. · Optical Ball, Bottle, Kitty, Ellipsoid test | PSNR: 22.57 -> 24.07; 22.59 -> 23.20; 18.76 -> 19.61; 25.21 -> 25.24 | Helped |
| Add differentiable transparent surface refinement to NeuS | NeuS+Ref. · Ball, Glass, Optical Ball, Bottle, Kitty, Ellipsoid test | LPIPS: 0.128 -> 0.105; 0.141 -> 0.120; 0.127 -> 0.112; 0.115 -> 0.084; 0.170 -> 0.163; 0.073 -> 0.061 | Helped |
| Add differentiable transparent surface refinement to NeuS | NeuS+Ref. · Ball and Glass test | PSNR: 21.15 -> 21.80; 20.15 -> 21.30 | Mixed |
| Add transparent surface refinement | NeuS · Optical Ball, Bottle, Kitty, Ellipsoid | Chamfer L1 (×10−3): 1.96 -> 1.91; 31.05 -> 29.54; 19.28 -> 18.05; 1.05 -> 0.80 | Helped |
Surprising findings
- NeuS+Ref. (analytic refraction without refinement) is worse than plain NeuS on Ball and Glass, showing an imperfect initial surface can hurt.
Limitations
- Only up to two refractions and one reflection are modeled, so complex objects like Kitty's ears are imperfect.
- Cannot model translucent objects; assumes scenes are fully transparent or opaque.
- Mixed results: PSNR/SSIM sometimes lower than baselines on Ball and Glass.
Experimental setup
Datasets: Glass (Bemana et al.) Ball (Bemana et al.) Kitty Optical Ball Bottle Ellipsoid
Benchmarks: Glass Ball Optical Ball Bottle Kitty Ellipsoid
Metrics: PSNR SSIM LPIPS chamfer-L1
Baselines: TensoRF Instant-NGP Nerfacto MS-NeRF Eikonal Fields NeuS NeuS+Ref. UNISURF VolSDF
Builds on: NeuS Gould et al. implicit differentiation (Proposition 4.6) sphere tracing SDFStudio
Closest papers by meaning
- From Transparent to Opaque: Rethinking Neural Implicit Surfaces with $\alpha$-NeuS (NeurIPS 2024 · similarity 0.90)
- Multiview Neural Surface Reconstruction by Disentangling Geometry and Appearance (NeurIPS 2020 · similarity 0.90)
- Opti-NeuS: Neural Reconstruction for Dual-Layered Transparent and Opaque Objects (CVPR 2026 · similarity 0.89)
- Seeing Through the Glass: Neural 3D Reconstruction of Object Inside a Transparent Container (CVPR 2023 · similarity 0.89)
- UniSDF: Unifying Neural Representations for High-Fidelity 3D Reconstruction of Complex Scenes with Reflections (NeurIPS 2024 · similarity 0.88)
- ProbeSDF: Light Field Probes For Neural Surface Reconstruction (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.