Research map / 3D vision / Depth and stereo
TurboSL: Dense Accurate and Fast 3D by Neural Inverse Structured Light
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Turns three- or four-pattern grayscale structured light into dense, sub-pixel accurate 3D by fitting a neural SDF with bidirectional volume rendering inverse-SL model, plus a no-reference error proxy.
Problem
Few-pattern structured-light decoding (three or four grayscale patterns with one camera) is noisy and yields outliers and poor sub-pixel disparity, limiting fast accurate 3D capture.
Approach
Formulates structured-light decoding as neural inverse rendering over the full image formation model (direct, ambient, indirect, projector blur) using a hybrid SDF-to-density volume rendering representation, jointly recovering disparity, normals, reflectance and residuals; uncertainty is estimated by comparing front-to-back and back-to-front scanned reconstructions.
Key results
- Three-pattern TurboSL reconstructs megapixel disparity with sub-pixel accuracy from images captured at up to 60 fps on a grayscale camera.
- No-reference proxy (front-to-back minus back-to-front disparity) correlates with ground-truth disparity error with coefficient 0.964-0.976 on a wedge test surface.
- Training for n=3 images takes 12 minutes and 10GB VRAM on one RTX6000 GPU.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add projector loss to the camera-loss-only setup (ablation) | TurboSL · Statue reconstruction, three a la carte patterns | reconstruction fidelity | Helped |
| add the cosine factor to the camera-loss-only setup (ablation) | TurboSL · Statue reconstruction, three a la carte patterns | reconstruction fidelity | Helped |
| add a learnable projector blur kernel (ablation) | TurboSL · Statue reconstruction, three a la carte patterns | image reconstruction error | Helped |
| increase the number of patterns from four to five (ablation) | TurboSL · Plush toy reconstruction | geometry accuracy | Helped |
| increase the number of patterns from five to six (ablation) | TurboSL · Plush toy reconstruction | geometry accuracy | Helped |
| use bidirectional rendering to compare front-to-back and back-to-front reconstructions (ablation) | TurboSL · Fabric animal reconstruction | identification of high-error regions | Helped |
| use three patterns rather than four (ablation) | TurboSL · Few-pattern structured-light reconstruction | geometry accuracy | Hurt |
Surprising findings
- Pixelwise ZNCC decoding with few patterns yields very poor meshes, while the same patterns decoded globally by TurboSL are far more accurate.
- 2D MLP representations fail to recover accurate geometry for inverse SL, and volumetric density fields give inaccurate surfaces despite reproducing inputs.
Limitations
- Assumes no semi-transparent surfaces and single-viewpoint, in-focus scene within depth of field.
- No-reference error proxy overestimates ground-truth error in some regions and relies on a conservative threshold.
- Indirect light integral is not modeled; residual term absorbs indirect and ambient contributions.
Experimental setup
Datasets: self-captured structured light images of real scenes (statue, plush toy, book, fruit bowl)
Metrics: disparity error (pixels) mean intensity error normal reconstruction quality
Baselines: ZNCC pixelwise decoder a la carte patterns MPS patterns Hamiltonian patterns 2D MLP decoder density field volumetric decoder
Builds on: NerfAcc NeuS ZNCC decoding a la carte structured light patterns volume rendering
Closest papers by meaning
- Real-time Acquisition and Reconstruction of Dynamic Volumes with Neural Structured Illumination (CVPR 2024 · similarity 0.87)
- Sparse Views Near Light: A Practical Paradigm for Uncalibrated Point-light Photometric Stereo (CVPR 2024 · similarity 0.87)
- Neural Inverse Rendering from Propagating Light (CVPR 2025 · similarity 0.87)
- Gated Fields: Learning Scene Reconstruction from Gated Videos (CVPR 2024 · similarity 0.86)
- NeLF-Pro: Neural Light Field Probes for Multi-Scale Novel View Synthesis (CVPR 2024 · similarity 0.86)
- Free360: Layered Gaussian Splatting for Unbounded 360-Degree View Synthesis from Extremely Sparse and Unposed Views (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.