Research map / 3D vision / Depth and stereo

RGB-Only Supervised Camera Parameter Optimization in Dynamic Scenes

NeurIPS 2025 · 3D vision › Depth and stereo · approach: Camera model and calibration for SfM · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ROS-Cam, an RGB-only supervised camera parameter optimization method for dynamic scenes that uses patch-wise tracking filters on CoTracker, outlier-aware Cauchy-uncertainty joint optimization over sparse calibration points, and a two-stage Softplus-based schedule.

Problem

Estimating per-frame camera parameters (focal length, rotation, translation) accurately and efficiently from casually captured dynamic-scene RGB videos without GT motion masks or other priors, where COLMAP is slow and fails on moving objects.

Approach

Builds sparse, texture-selected point tracks from a pre-trained point tracker alone, attaches a learnable Cauchy-distributed uncertainty to each 3D calibration point (rather than per-pixel) to down-weight moving outliers, and uses a two-stage schedule (fixed uncertainty then jointly optimized) motivated by Softplus asymptotics and the convex inner term of the Cauchy loss.

Key results

• On MPI-Sintel, the method achieves lower ATE/RPE than casualSAM on most of the 18 videos (e.g., market_6 ATE 0.009 vs 0.239).
• On iPhone NVS with 4DGS, the method is best among RGB-only methods and sometimes beats LiDAR-based Record3D (e.g., Haru-sit PSNR 28.31 vs 29.86 for Record3D, Handwavy 28.02 vs 27.80).
• Average camera-optimization runtime on DAVIS is 2.68 min versus 31 min for COLMAP w/o mask and 17 min for casualSAM.

Measured effects reported in the paper

Surprising findings

• The LiDAR-based Record3D 'ground truth' camera parameters on iPhone are occasionally unreliable, and the RGB-only method sometimes outperforms them in NVS.
• COLMAP with or without motion masks fails on several MPI-Sintel and DAVIS videos, so comparisons exclude it.

Limitations

• Assumes constant focal length, so zoom videos are not handled.
• Performs poorly when large moving objects dominate the screen, as most tracked trajectories then belong to outliers (e.g., MPI-Sintel failure cases).
• Relies on pre-trained point tracker quality; struggles on some special cases such as low-parallax or large-object scenes.

Experimental setup

Datasets: NeRF-DS DAVIS iPhone MPI-Sintel TUM-dynamics

Benchmarks: NeRF-DS DAVIS iPhone MPI-Sintel TUM-dynamics

Metrics: ATE RPE trans RPE rot PSNR SSIM LPIPS runtime

Baselines: COLMAP w/o mask COLMAP w/ mask casualSAM Robust-CVD Record3D RoDynRF

Builds on: CoTracker COLMAP 4DGS 3DGS

Closest papers by meaning

• RGBD Objects in the Wild: Scaling Real-World 3D Object Learning from RGB-D Videos (CVPR 2024 · similarity 0.87)
• Motion4D: Learning 3D-Consistent Motion and Semantics for 4D Scene Understanding (NeurIPS 2025 · similarity 0.86)
• 4D3R: Motion-Aware Neural Reconstruction and Rendering of Dynamic Scenes from Monocular Videos (NeurIPS 2025 · similarity 0.86)
• VGGT-SLAM: Dense RGB SLAM Optimized on the SL(4) Manifold (NeurIPS 2025 · similarity 0.86)
• Joint Optimization of Neural Radiance Fields and Continuous Camera Motion from a Monocular Video (CVPR 2025 · similarity 0.85)
• OSN: Infinite Representations of Dynamic 3D Scenes from Monocular Videos (ICML 2024 · similarity 0.85)