Research map / 3D vision / Depth and stereo
FNIN: A Fourier Neural Operator-based Numerical Integration Network for Surface-from-gradients
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FNIN, a Fourier neural operator-based numerical integration network with a two-stage pipeline (learned FNO integration with self-learned discontinuity attention, then one-step weighted least squares refinement) for surface-from-gradients.
Problem
Recovering a 3D surface from its gradients (surface-from-gradients / normal integration) is hard because of discontinuities, noise, integrability violations, and the cost of large-scale linear solvers at high resolution.
Approach
Decouple smooth-area integration from discontinuity handling: an iterative Fourier neural operator integrates normals in Fourier space while a self-learned attention map, trained with a detail-weighted loss, down-weights discontinuities; a single weighted least squares solve then determines depth jumps.
Key results
- On LUCES (2048x1536), FNIN-S achieves a mean-of-objects-level improvement over classical solvers with per-object MAE often below 0.3 mm (e.g., Ball 0.15 mm, Bell 0.12 mm, Cup <0.01 mm) and runtime about 1.3 s versus over 6000 s for HB.
- On DiLiGenT, FNIN-S reports lower MAE than Rec-Net, which fails numerically on several objects, and achieves 0.11 mm on Bear vs. 0.89 mm for the best baseline IPF.
- Ablations show removing the attention network increases LUCES MAE from 1.92 to 4.02 mm and removing discontinuity optimization raises LUCES MAE from 1.92 to 3.00 mm.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the FNIN integration network rather than using BiNI (k=0) (ablation) | FNIN · DiLiGenT (10 objects) | MAE (mm): 4.59 -> 4.23 | Helped |
| Add the FNIN integration network rather than using BiNI (k=0) (ablation) | FNIN · LUCES (14 objects) | MAE (mm): 2.79 -> 1.88 | Helped |
| Add attention (ablation) | FNIN · DiLiGenT (10 objects) | MAE (mm): 5.97±0.35 -> 4.5456±0.0083 | Helped |
| Add attention (ablation) | FNIN · LUCES (14 objects) | MAE (mm): 4.20±0.42 -> 1.9246±0.0004 | Helped |
| Use attention instead of mask concatenation (ablation) | FNIN · DiLiGenT (10 objects) | MAE (mm): 17.78±2.26 -> 4.5456±0.0083 | Helped |
| Use attention instead of mask concatenation (ablation) | FNIN · LUCES (14 objects) | MAE (mm): 4.18±0.42 -> 1.9246±0.0004 | Helped |
| Add discontinuity optimization (ablation) | FNIN · DiLiGenT (10 objects) | MAE (mm): 5.83±0.19 -> 4.5456±0.0083 | Helped |
| Add discontinuity optimization (ablation) | FNIN · LUCES (14 objects) | MAE (mm): 3.05±0.06 -> 1.9246±0.0004 | Helped |
| Use 1 Fourier layer instead of 4 (ablation) | FNIN · DiLiGenT | MAE: 5.91 -> 6.38 | Hurt |
| Use 1 Fourier layer instead of 4 (ablation) | FNIN · LUCES | MAE: 3.00 -> 4.40 | Hurt |
| Use 10 Fourier layers instead of 4 (ablation) | FNIN · DiLiGenT | MAE: 5.91 -> 6.03 | Helped |
| Use fewer than 16 truncated Fourier modes (ablation) | FNIN · DiLiGenT and LUCES | MAE: 5.91 -> 8.11; 3.00 -> 4.83 | Hurt |
Surprising findings
- Traditional and learning baselines fail numerically on several objects (e.g., HB, DF, Rec-Net), and Rec-Net fails entirely on DiLiGenT.
- Attention and mask concatenation increase error and variance because they produce uncertain jumps at discontinuities.
Limitations
- Surface islands from scale or offset ambiguity remain unresolved.
- Relative discontinuity weight depends on training data, which lacks fine detail, so large discontinuities may be under-weighted.
- Extension to nonlinear equations (near-field photometric stereo) remains open.
Experimental setup
Datasets: synthetic statue dataset (14 objects, rendered) DiLiGenT LUCES
Benchmarks: DiLiGenT LUCES
Metrics: Mean Absolute Error (mm) runtime (s) RAM+VRAM (GB)
Baselines: Horn and Brooks (HB) FFT integration DCT DST discretized functional (DF) Savitzky-Golay variational (ZS) inverse plane fitting (IPF) L1 anisotropic diffusion (AD) BiNI Rec-Net WLS
Builds on: Fourier neural operator Rec-Net BiNI weighted least squares (Quéau and Durou)
Closest papers by meaning
- Flatten Anything: Unsupervised Neural Surface Parameterization (NeurIPS 2024 · similarity 0.85)
- SuperNormal: Neural Surface Reconstruction via Multi-View Normal Integration (CVPR 2024 · similarity 0.84)
- Neural Operators with Localized Integral and Differential Kernels (ICML 2024 · similarity 0.84)
- GSO-Net: Grid Surface Optimization via Learning Geometric Constraints (AAAI 2024 · similarity 0.84)
- Discontinuity-preserving Normal Integration with Auxiliary Edges (CVPR 2024 · similarity 0.84)
- NeuSurf: On-Surface Priors for Neural Surface Reconstruction from Sparse Input Views (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.