Research map / 3D vision / Depth and stereo

BA-T: An Iterative Transformer for Two-View Bundle Adjustment

NeurIPS 2026 · 3D vision › Depth and stereo · approach: Cost-volume multi-view stereo · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes BA-T, a single reusable bundle-adjustment-inspired transformer layer that iteratively refines camera and local geometry tokens in latent space for two-view (and extended multi-view) feed-forward 3D reconstruction.

Problem

Feed-forward 3D reconstruction models rely on deep cross-view attention decoder stacks and lack a structured, iterative geometry refinement mechanism, leading to weaker multi-view consistency.

Approach

Models classical bundle adjustment's information flow between camera and geometry as a repeatable learned layer operating on latent tokens: camera-conditioned geometry transform, soft token-level correspondence, latent residuals, then camera update followed by geometry update, applied iteratively with iteration-weighted supervision.

Key results

• BA-T (K=4) reaches 0.683 rotation AUC@5° on 7Scenes versus 0.677 for the best iterative-decoder baseline and reduces 3D correspondence error by 44% across iterations versus 27% for ViSTA† with iterative training.
• BA-T uses a 38M-parameter decoder, versus 113M–765M for prior feed-forward models, reaching 16% of DUSt3R's decoder parameters while matching or surpassing them on pose metrics.
• On 4-view BundleFusion, BA-T achieves 50.62 ms inference and 1.35 GB peak memory, about 39% of MapAnything's memory, with accuracy comparable to prior methods.

Surprising findings

• Classical Droid-SLAM bundle adjustment, initialized from BA-T iteration 0 with ground-truth intrinsics, underperforms BA-T in both pose and geometry accuracy, attributed to non-convex optimization under limited co-visibility.

Limitations

• Current implementation and training focus on two-view reconstruction; multi-view extension is only lightly evaluated (4 views on BundleFusion).
• Geometry gains are more limited than pose gains across iterations.
• Per-view geometry is combined with poses rather than optimized as a single global representation.

Experimental setup

Datasets: ScanNet ScanNet++ ARKitScenes CO3D ASE Replica

Benchmarks: 7Scenes BundleFusion TUM-RGBD

Metrics: AUC@5/10/20 (pose) translation AUC Chamfer distance accuracy completeness delta 1.25 relative depth error 3D correspondence error ATE inference time

Baselines: VGGT MapAnything Depth Anything 3 DUSt3R MASt3R ViSTA ViSTA† DROID-SLAM

Builds on: DUSt3R DiT ViSTA Schur complement bundle adjustment

Closest papers by meaning

• Déjà View: Looping Transformers for Multi-View 3D Reconstruction (NeurIPS 2026 · similarity 0.86)
• Understanding Multi-View Transformers (NeurIPS 2026 · similarity 0.85)
• 4D3R: Motion-Aware Neural Reconstruction and Rendering of Dynamic Scenes from Monocular Videos (NeurIPS 2025 · similarity 0.85)
• TurboVGGT: Fast Visual Geometry Reconstruction with Adaptive Alternating Attention (NeurIPS 2026 · similarity 0.84)
• PF3plat: Pose-Free Feed-Forward 3D Gaussian Splatting for Novel View Synthesis (ICML 2025 · similarity 0.84)
• Sparse-view Pose Estimation and Reconstruction via Analysis by Generative Synthesis (NeurIPS 2024 · similarity 0.84)