Research map / 3D vision / Depth and stereo
From a Bird's Eye View to See: Joint Camera and Subject Registration without the Camera Calibration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a calibration-free framework that jointly registers multi-view cameras and pedestrians in a virtual bird's-eye view from first-person RGB images, with a new synthetic dataset (CSRD) and a real cross-domain test set.
Problem
Multi-view camera and subject (pedestrian) registration in a bird's eye view without camera calibration or a real BEV image, using only several first-person-view RGB images.
Approach
Alternates subject and camera registration: a view-transform LocoNet projects each pedestrian's 2D pose to BEV position and orientation; a geometry-based spatial alignment module estimates relative camera poses from matched subject pairs; a centroid-based selection picks the camera pose, which is then used to fuse subjects and self-supervise the appearance-based association network.
Key results
- On CSRD-II, camera registration achieves mean position error 0.89 m and mean orientation error 5.78 degrees, versus 3.00 m / 21.84 degrees for Monoloco++.
- Self-supervised subject association raises cross-view F1 from 66.78% (ResNet-50 baseline) to 85.98%, close to the oracle 86.43%.
- On real CSRD-R, the synthetic-trained model reaches 82.50%, 85.07%, and 86.31% for two, three, and four views.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LocoNet pre-training (ablation) | camera and subject registration on CSRD-II · CSRD-II camera registration | Cam.Pos.Avg (m): 6.98 -> 0.89 | Helped |
| Remove camera orientation supervision (ablation) | camera and subject registration on CSRD-II · CSRD-II camera registration | Cam.Ori.Avg (degrees): 5.91 -> 5.78 | No clear effect |
| Select the centroid-nearest camera-pose candidate (ablation) | camera registration on CSRD-II · CSRD-II camera registration | Cam.Pos.Avg (m): 2.27 -> 0.89; 1.91 -> 0.89 | Helped |
| Use LocoNet pre-training (ablation) | subject registration on CSRD-II · CSRD-II subject registration | Sub.Pos.Avg (m): 6.35 -> 0.75 | Helped |
| Remove camera orientation supervision (ablation) | subject registration on CSRD-II · CSRD-II subject registration | Sub.Ori.Avg (degrees): 16.36 -> 14.67 | Helped |
| Select the centroid-nearest camera-pose candidate (ablation) | subject registration on CSRD-II · CSRD-II subject registration | Sub.Pos.Avg (m): 1.27 -> 0.75; 1.06 -> 0.75 | Helped |
| Train appearance association with the proposed self-supervised strategy | ResNet-50 subject association · CSRD-II subject association | F1: 66.78% -> 85.98% | Helped |
| Add subject-matching constraints (ablation) | multi-view subject registration · CSRD-V multi-view registration | Sub.Pos.Avg (m): 1.10 -> 0.94 | Helped |
| Add subject-matching constraints (ablation) | multi-view subject association · CSRD-V multi-view subject association | F1: 85.64% -> 86.12% | Helped |
Surprising findings
- Removing the ground-truth camera orientation supervision only slightly degrades camera registration (mean orientation error 5.91 vs 5.78 degrees), so the method is not heavily dependent on orientation labels.
Limitations
- Subject registration relies on pedestrian 2D skeletons from PifPaf and on appearance-based matching, which can produce wrong fusion from incorrect matching.
- Camera registration is evaluated on a synthetic city scene of 25m x 25m; the real-world evaluation has no quantitative camera-pose ground truth reported in the main text.
Experimental setup
Datasets: CSRD-II CSRD-V CSRD-R Market-1501
Benchmarks: CSRD-II CSRD-V CSRD-R
Metrics: average position error (m) average orientation error (deg) percentage within distance/angle thresholds precision recall F1
Baselines: Monoloco++ DMHA SIFT+KNN LoFTR SuperGlue CVNet R2Former MVDet MVDetr
Builds on: PifPaf ResNet-50 Monoloco++ MVDet
Closest papers by meaning
- DynPose: Largely Improving the Efficiency of Human Pose Estimation by a Simple Dynamic Framework (CVPR 2025 · similarity 0.79)
- From Correspondences to Pose: Non-minimal Certifiably Optimal Relative Pose without Disambiguation (CVPR 2024 · similarity 0.79)
- UNOPose: Unseen Object Pose Estimation with an Unposed RGB-D Reference Image (CVPR 2025 · similarity 0.78)
- GoMVS: Geometrically Consistent Cost Aggregation for Multi-View Stereo (CVPR 2024 · similarity 0.78)
- Optimizing Human Pose Estimation Through Focused Human and Joint Regions (AAAI 2025 · similarity 0.78)
- UniCon3R: Unified Contact-aware 4D Human-Scene Reconstruction from Monocular Video (NeurIPS 2026 · similarity 0.78)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.