Research map / 3D vision / Depth and stereo
Rethinking Inductive Biases for Surface Normal Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DSINE, a surface normal estimator that encodes per-pixel ray directions as input, applies a ray-direction ReLU activation, and refines normals by iteratively learning pairwise relative rotations between neighboring pixels, trained on a 160K-image meta-dataset.
Problem
Surface normal estimation from a single RGB image is typically handled by general-purpose dense prediction models that reuse inductive biases from other tasks (translation-equivariant CNNs, transformers with positional encodings), which ignore camera intrinsics and inter-pixel normal constraints and
Approach
Injects task-specific inductive biases: dense focal-length-normalized ray directions as network input, a ray-based ReLU that keeps predicted normals visible, and an iterative update that rotates neighboring normals via predicted axis-angle rotations (ConvGRU-based, convex upsampling).
Key results
- On Sintel (out-of-distribution FoV and aspect ratio), the method reaches 34.9 mean angular error vs 40.5 for Omnidata v2.
- On Virtual KITTI, mean error 28.9 vs 33.0 for Omnidata v2 with 11.25° accuracy 9.9 vs 4.6 hmm.
- Uses 72M parameters vs 123M for Omnidata v2 and trains in 12 hours on a single RTX 4090.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Ours instead of Omnidata v2 | surface normal estimation · Sintel | mean angular error: 40.5 -> 34.9 | Helped |
| use Ours instead of Omnidata v2 | surface normal estimation · Virtual KITTI | mean angular error: 37.5 -> 28.9 | Helped |
| add per-pixel ray direction as input (ablation) | base surface normal model · Sintel | mean angular error: 36.6 -> 36.0 | Helped |
| add per-pixel ray direction as input (ablation) | base surface normal model · Virtual KITTI | mean angular error: 30.5 -> 29.4 | Helped |
| add per-pixel ray direction as input (ablation) | base surface normal model · NYUv2 | mean angular error: 16.6 -> 16.6 | No clear effect |
| add iterative rotation estimation (ablation) | base+ray surface normal model · Sintel | mean angular error: 36.0 -> 34.9 | Helped |
| add iterative rotation estimation (ablation) | base+ray surface normal model · Virtual KITTI | mean angular error: 29.4 -> 28.9 | Helped |
| add iterative rotation estimation (ablation) | base+ray surface normal model · iBims-1 | mean angular error: 17.6 -> 17.1 | Helped |
| remove ray ReLU activation (ablation) | Ours · Virtual KITTI | mean angular error: 28.9 -> 29.6 | Hurt |
| remove ray ReLU activation (ablation) | Ours · ScanNet | mean angular error: 16.2 -> 16.1 | No clear effect |
Surprising findings
- Rotation refinement yields only small gains on standard accuracy metrics, which are dominated by large planar surfaces; benefits show mainly qualitatively near boundaries.
- Despite being trained on about 160K images versus 12M for Omnidata, the method performs on par on in-distribution-like datasets such as NYUv2 and ScanNet.
Limitations
- Requires known or approximated camera intrinsics; intrinsics approximated from metadata for in-the-wild OASIS images.
- Accuracy metrics on OASIS are unreliable because ground truth covers only small human-annotated flat patches.
Experimental setup
Datasets: Cleargrasp 3D Ken Burns Hypersim SAIL-VOS 3D TartanAir MVS-Synth BlendedMVG Taskonomy Replica Replica + GSO NYUv2 ScanNet
Benchmarks: NYUv2 ScanNet iBims-1 Sintel Virtual KITTI OASIS
Metrics: mean angular error median angular error percentage of pixels within 5.0, 7.5, 11.25, 22.5, 30 degree
Baselines: OASIS (Hourglass) EESNU Omnidata v1 Omnidata v2
Builds on: RAFT CAM-Convs Omnidata EfficientNet convex upsampling
Closest papers by meaning
- Normal-NeRF: Ambiguity-Robust Normal Estimation for Highly Reflective Scenes (AAAI 2025 · similarity 0.86)
- VCR-GauS: View Consistent Depth-Normal Regularizer for Gaussian Surface Reconstruction (NeurIPS 2024 · similarity 0.84)
- Normal-GS: 3D Gaussian Splatting with Normal-Involved Rendering (NeurIPS 2024 · similarity 0.84)
- M-plicits: Neural Implicit Surfaces via Nested Multiscale Residuals (NeurIPS 2026 · similarity 0.83)
- OffsetOPT: Explicit Surface Reconstruction without Normals (CVPR 2025 · similarity 0.83)
- PMNI: Pose-free Multi-view Normal Integration for Reflective and Textureless Surface Reconstruction (CVPR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.