Research map / Point clouds / Registration and matching
ColabSfM: Collaborative Structure-from-Motion by Point Cloud Registration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes registering SfM reconstructions as a 3D point cloud registration problem, with a synthetic trajectory-based dataset generation pipeline from MegaDepth and an improved RoITr variant (RefineRoITr) that refines correspondences with a local transformer.
Problem
Merging or registering distributed Structure-from-Motion reconstructions into a joint reference frame without visual descriptors, given a lack of scalable methods and training datasets for SfM point cloud registration.
Approach
Treat SfM map merging as geometry-only point cloud registration (3D points and normals, no descriptors), generate training pairs by retriangulating partial reconstructions from synthetic camera trajectories sampled from MegaDepth scenes, and add a local refinement transformer on top of the rotation-invariant RoITr.
Key results
- RefineRoITr trained from scratch on the proposed dataset reaches 70.2% registration recall and 96.5% FMR on MegaDepth SE(3), versus 3.0% IR / 0.0% RR for RoITr trained on 3DMatch.
- On Cambridge Landmarks, RefineRoITr (Mega) reaches inlier ratios of 70.9% (Great Court) and 81.8% (St Mary's Church), far above 3DMatch-only baselines.
- Models trained only on 3DMatch fail to register SfM point clouds (near-zero inlier ratios on most scenes), while the proposed dataset yields large gains; runtime overhead of refinement is about 3% (102.84 ms vs 99.94 ms).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the refinement Transformer to RoITr (ablation) | RoITr trained on 3DMatch · MegaDepth SE(3) | RR: 0.0 -> 13.9 | Helped |
| train on the MegaDepth SfM registration dataset instead of 3DMatch | RefineRoITr · MegaDepth SE(3) | RR: 13.9 -> 70.2 | Helped |
| train on the MegaDepth SfM registration dataset instead of 3DMatch | RefineRoITr · MegaDepth Sim(3) | RR: 3.5 -> 42.7 | Helped |
| add the refinement Transformer to RoITr (ablation) | RoITr trained on 3DMatch and fine-tuned on MegaDepth · MegaDepth SE(3) | RR: 60.0 -> 67.7 | Helped |
| add the refinement Transformer to RoITr (ablation) | RoITr trained on 3DMatch and fine-tuned on MegaDepth · MegaDepth Sim(3) | RR: 38.7 -> 44.3 | Helped |
| fine-tune on the MegaDepth SfM registration dataset | RefineRoITr · Quad6k | RR: 2.0 -> 22.4 | Helped |
| fine-tune on the MegaDepth SfM registration dataset | RefineRoITr · Quad6k | RR: 2.0 -> 22.4; 2.0 -> 16.3 | Mixed |
| train from scratch on MegaDepth rather than fine-tune from 3DMatch | RefineRoITr · MegaDepth SE(3) | RR: 67.7 -> 70.2 | Helped |
| train from scratch on MegaDepth rather than fine-tune from 3DMatch | RefineRoITr · Quad6k | RR: 22.4 -> 16.3 | Hurt |
Surprising findings
- Models trained on 3DMatch, including the state-of-the-art RoITr, essentially fail to register SfM point clouds out of the box, with near-zero registration recall.
Limitations
- Model is E(3)/SE(3)-invariant like RoITr and can struggle with symmetric scenes.
- Training and evaluation use retriangulated partial reconstructions with fixed camera poses rather than real drift-affected reconstructions.
Experimental setup
Datasets: MegaDepth 3DMatch Cambridge Landmarks 7-Scenes Quad6k
Benchmarks: MegaDepth SfM registration benchmark Cambridge Landmarks 7-Scenes Quad6k
Metrics: Inlier Ratio Feature Matching Recall Registration Recall number of matches
Baselines: RoITr OverlapPredator GeoTransformer PareNet
Builds on: RoITr GeoTransformer point pair features hloc COLMAP
Closest papers by meaning
- VGGSfM: Visual Geometry Grounded Deep Structure From Motion (CVPR 2024 · similarity 0.88)
- Light3R-SfM: Towards Feed-forward Structure-from-Motion (CVPR 2025 · similarity 0.86)
- MP-SfM: Monocular Surface Priors for Robust Structure-from-Motion (CVPR 2025 · similarity 0.86)
- Reconstructing People, Places, and Cameras (CVPR 2025 · similarity 0.85)
- PosDiffNet: Positional Neural Diffusion for Point Cloud Registration in a Large Field of View with Perturbations (AAAI 2024 · similarity 0.85)
- Dense-SfM: Structure from Motion with Dense Consistent Matching (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.