Research map / 3D vision / Human pose estimation
Dynamic Camera Poses and Where to Find Them
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds DynPose-100K, a 100K-video dataset of dynamic Internet videos with camera poses, via a VLM-and-specialist filtering pipeline over Panda-70M plus a masking/point-tracking/global-SfM pose estimator.
Problem
Camera pose annotation for dynamic Internet videos at scale is hard because most such videos are unsuitable for pose estimation and standard SfM fails on moving objects and appearance changes.
Approach
Combine task-specific specialist filters and a generalist VLM (GPT-4o mini) to select pose-estimable dynamic videos, then estimate poses by stacking dynamic masking (OneFormer, Hands23, motion masking, SAM2), BootsTAP sliding-window point tracking, and Theia-SfM global bundle adjustment.
Key results
- On Lightspeed, the proposed pipeline cuts ATE by 50% across all videos and 90% on easy sequences versus the best baseline (MonST3R).
- On Panda-Test, the pipeline registers 95.6% of videos with mean reprojection error 5.76 px vs 6.78 for DROID-SLAM and 9.86 for MonST3R.
- Filtering reaches test precision 0.78 at the operating threshold, a level no baseline achieves at any recall except reprojection error at 0.02 recall.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the proposed tracking, dynamic masking, and global bundle adjustment pipeline instead of MonST3R | Dynamic camera pose estimation · Lightspeed, all 36 videos | ATE (m): 0.149 -> 0.072 | Helped |
| Use the proposed pose estimation pipeline instead of LEAP-VO | Dynamic camera pose estimation · Panda-Test, all 90 estimable videos | mean per-video reprojection error (720p): 7.59 -> 5.76 | Helped |
| Fine-tune DUSt3R on DynPose-100K instead of synthetic data | DUSt3R · Panda-Test | mean per-video reprojection error (720p): 9.86 -> 8.78 | Helped |
| Add optical-flow filtering (ablation) | Video filtering pipeline · Panda-Test | precision and recall | Helped |
| Add point-tracking filtering (ablation) | Video filtering pipeline · Panda-Test | precision and recall | Helped |
| Add dynamic-mask filtering (ablation) | Video filtering pipeline · Panda-Test | precision and recall | Helped |
| Add focal-length filtering (ablation) | Video filtering pipeline · Panda-Test | precision and recall | Helped |
| Add distortion filtering (ablation) | Video filtering pipeline · Panda-Test | precision and recall | Helped |
| Add VLM filtering (ablation) | Video filtering pipeline · Panda-Test | precision and recall | Helped |
Surprising findings
- Only 9% of randomly sampled Panda-70M videos are suitable for dynamic camera pose estimation.
- Fine-tuning DUSt3R on 2K DynPose-100K videos gives lower mean error than fine-tuning on MonST3R's synthetic data.
Limitations
- Only about 9% of Internet videos pass the suitability criteria, so filtering discards most candidates.
- Evaluation on Internet videos relies on sparse manually annotated correspondences rather than ground-truth poses.
- Pose pipeline relies on several separate pretrained components (masking, tracking, SfM) that are not jointly trained.
Experimental setup
Datasets: Panda-70M DynPose-100K Lightspeed Panda-Test
Benchmarks: Lightspeed Panda-Test
Metrics: ATE RPE translation RPE rotation precision recall Sampson reprojection error percent videos registered
Baselines: ParticleSfM COLMAP DROID-SLAM DUSt3R MonST3R LEAP-VO CamCo B-Timer
Builds on: ParticleSfM BootsTAP Theia-SfM Panda-70M CamCo
Closest papers by meaning
- DynaTokens: Teaching Dynamics to Camera-Controlled Video Models at Test Time (NeurIPS 2026 · similarity 0.87)
- MegaSaM: Accurate, Fast and Robust Structure and Motion from Casual Dynamic Videos (CVPR 2025 · similarity 0.86)
- ProxyPose: 6-DoF Pose Tracking via Video-to-Video Translation (NeurIPS 2026 · similarity 0.86)
- SpatioTemporal Learning for Human Pose Estimation in Sparsely-Labeled Videos (AAAI 2025 · similarity 0.86)
- HumanVid: Demystifying Training Data for Camera-controllable Human Image Animation (NeurIPS 2024 · similarity 0.85)
- DyST: Towards Dynamic Neural Scene Representations on Real-World Videos (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.