Research map / 3D vision / Depth and stereo

World Tracing: Pixel-Aligned Geometry Beyond the Visible

NeurIPS 2026 · 3D vision › Depth and stereo · approach: Feed-forward 3D reconstruction

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes World Tracing, a pixel-aligned multilayer camera-space pointmap representation generated by a flow-matching diffusion transformer (WT-DiT) that completes occluded geometry while keeping visible-surface accuracy.

Problem

Single-image 3D methods either stay pixel-aligned but stop at the visible surface (depth/pointmap predictors) or generate complete shapes in a canonical frame that is misaligned with the input image.

Approach

Predict an ordered stack of L camera-space 3D points per input pixel, supervised by forward-filled dense depth-peeled targets (no per-layer masks), with flow matching over the image grid, three-way attention (layer-wise, ray-wise, global), and a layer-aware mixed noise schedule.

Key results

• WT-O visible-surface AbsRel 0.0079 vs 0.0138 for VGGT and 0.0141 for MoGe-2 on held-out objects.
• WT-O full-object L2 0.00194 and F@0.05 0.898 vs best baseline F@0.05 0.677 (ReconViaGen/LaS-Comp).
• WT-D mean global CD-L2 0.0105 vs 0.0162 for ActionMesh on dynamic benchmarks; ActionMesh is best on ActionBench (0.0243 vs WT-D 0.0291).

Surprising findings

• AbsRel-based depth gains on objects are large even though deeper layers are generated, suggesting multilayer generation does not hurt visible-surface accuracy.
• Mask-based per-layer prediction collapses due to severe imbalance (mean valid area 8.14% at L0 to 0.60% at L5), motivating depth filling.

Limitations

• Dynamic model is outperformed by ActionMesh on the ActionBench split.
• Reported results use best-of-8 seeds for generative methods, which favors stochastic methods relative to single-sample evaluation.
• Scene generalization is evaluated mainly on a small internal test set and outdoor performance lags on some scenes.

Experimental setup

Datasets: Objaverse-XL Objaverse 3D-FUTURE Toys4k GSO TrueBones 3D-FRONT ScanNet v2 MegaDepth BlendedMVS ArkitScenes Argoverse2

Benchmarks: held-out object assets 3D-FRONT held-out internal scene test set Obj.-Val Truebone ActionBench NYU Depth V2 ETH3D DAVIS

Metrics: MAE RMSE AbsRel delta<1.25 Chamfer distance L1/L2 F-score CD-L2

Baselines: DA3 LaRI Pi3X MoGe-2 VGGT TRELLIS.2 SAM 3D LaS-Comp ReconViaGen GVFDiffusion SS4D ActionMesh

Builds on: flow matching MoGe DiT LaRI VGGT TRELLIS

Closest papers by meaning

• G4Splat: Geometry-Guided Gaussian Splatting with Generative Prior (ICLR 2026 · similarity 0.89)
• VA-GS: Enhancing the Geometric Representation of Gaussian Splatting via View Alignment (NeurIPS 2025 · similarity 0.88)
• NOVA3R: Non-pixel-aligned Visual Transformer for Amodal 3D Reconstruction (ICLR 2026 · similarity 0.88)
• World from Motion: Generative Dynamic Gaussian Reconstruction from Monocular Video (NeurIPS 2026 · similarity 0.87)
• Coherent 3D Scene Diffusion From a Single RGB Image (NeurIPS 2024 · similarity 0.87)
• Tango3D: Towards Alignment for Global and Local 2D-3D Correspondence (NeurIPS 2026 · similarity 0.87)