Research map / Video / Video segmentation and tracking

BAT3R: Robust Online 3D Reconstruction with Bayesian Adaptive State Updates

NeurIPS 2026 · Video › Video segmentation and tracking · approach: Stochastic video prediction

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Training-free Bayesian adaptation of the CUT3R/TTT3R recurrent state for streaming 3D reconstruction, combining anchor-based gating (SIG) with a Kalman-filter state update (BASU) weighted by confidence and cross-attention visibility.

Problem

Streaming recurrent 3D reconstruction models accumulate drift and degrade over long video sequences because state updates lack principled uncertainty handling and anchoring to the initial state.

Approach

Reformulates the recurrent state update of TTT3R as a per-token Kalman filter, using the decoder's confidence maps as observation noise and cross-attention visibility as spatial relevance, plus an anchor-based innovation gate that pulls drifted tokens back toward the previous state, all without training.

Key results

• On 1000-frame sequences BAT3R reduces ATE by 67.8% on ScanNet and 66.1% on TUM dynamics relative to CUT3R, and by 32.3% and 44.4% relative to TTT3R (as stated in the text).
• On 7-Scenes at 400 frames, Chamfer distance drops from 0.134 (CUT3R) to 0.024, and from 0.039 to 0.024 versus TTT3R (as stated in the text).
• In short-sequence TUM-dynamics evaluation, BAT3R achieves ATE 0.016 vs 0.028 for TTT3R and 0.046 for CUT3R.

Surprising findings

• On short-sequence Sintel camera pose, BAT3R (ATE 0.219) is worse than TTT3R (0.201) and CUT3R (0.213), showing gains are concentrated on long sequences.

Limitations

• Gains on short sequences are limited; BAT3R trails TTT3R on Sintel short-sequence ATE.
• Remaining gap to strong offline methods such as VGGT on short sequences.
• Several hyperparameters (eta, kappa, tau, alpha, omega, gamma) are heuristically tuned on TUM-dynamics.

Experimental setup

Datasets: TUM dynamics ScanNet Sintel Bonn KITTI 7-Scenes NRGBD

Benchmarks: TUM dynamics ScanNet Sintel Bonn KITTI 7-Scenes NRGBD

Metrics: ATE RPE trans RPE rot Abs Rel delta<1.25 Chamfer distance accuracy completeness normal consistency

Baselines: CUT3R TTT3R TTSA3R Point3R StreamVGGT Spann3R VGGT DUSt3R

Builds on: CUT3R TTT3R test-time training Kalman filter

Closest papers by meaning

• Rethinking the State Update Gate for Long-Sequence Recurrent 3D Reconstruction (NeurIPS 2026 · similarity 0.90)
• Continuous 3D Perception Model with Persistent State (CVPR 2025 · similarity 0.88)
• 4D3R: Motion-Aware Neural Reconstruction and Rendering of Dynamic Scenes from Monocular Videos (NeurIPS 2025 · similarity 0.87)
• Living Scenes: Multi-object Relocalization and Reconstruction in Changing 3D Environments (CVPR 2024 · similarity 0.86)
• ViDAR: Video Diffusion-Aware 4D Reconstruction From Monocular Inputs (NeurIPS 2025 · similarity 0.86)
• Adaptive 3D Reconstruction via Diffusion Priors and Forward Curvature-Matching Likelihood Updates (NeurIPS 2025 · similarity 0.86)