Research map / 3D vision / Depth and stereo
Detector-Free Structure from Motion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a detector-free structure-from-motion pipeline that builds a coarse SfM model from quantized detector-free matches and then iteratively refines tracks with a transformer-based multi-view matcher and geometric bundle adjustment.
Problem
Traditional SfM relies on repeatable keypoint detection across views, which fails in texture-poor scenes; detector-free matchers produce pair-dependent, inconsistent locations that break multi-view feature tracks.
Approach
Quantize detector-free coarse matches into a grid to build a consistent coarse SfM model, then iterate between an attention-based multi-view feature transformer that refines feature tracks (with reference-view selection and location search) and geometric bundle adjustment with track topology adjustment.
Key results
- On the texture-poor SfM dataset, the LoFTR-based framework reaches AUC@5° of 35.77 versus 30.49 for LoFTR+PixSfM and 19.23 for SP+SG+PixSfM.
- On ETH3D triangulation at 1cm, Ours (LoFTR) achieves 80.24% accuracy and 3.73% completeness, versus 79.01% and 0.75% for SP+SG+PixSfM.
- Geometric BA memory on a 2000-image scene is 2.63 GB versus 904.5 GB for PixSfM with feature maps and 21.2 GB with cost maps.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use the detector-free SfM framework with LoFTR matches | Ours (LoFTR) · ETH3D | AUC@1°: 50.82 -> 59.12 | Helped |
| use the detector-free SfM framework with LoFTR matches | Ours (LoFTR) · IMC | AUC@3°: 46.30 -> 46.94 | Helped |
| use the detector-free SfM framework with LoFTR matches | Ours (LoFTR) · Texture-Poor SfM Dataset | AUC@3°: 20.66 -> 26.07 | Helped |
| use quantization ratio r = 4 instead of r = 8 (ablation) | Ours (LoFTR) · ETH3D | Accuracy (1cm): 80.24 -> 81.58 | Helped |
| use quantization ratio r = 2 instead of r = 8 (ablation) | Ours (LoFTR) · ETH3D | Accuracy (1cm): 80.24 -> 81.18 | Helped |
| remove iterative refinement (ablation) | Ours (LoFTR) · ETH3D | Accuracy (1cm): 80.24 -> 42.13 | Hurt |
| use 1 refinement iteration instead of 2 (ablation) | Ours (LoFTR) · ETH3D | Accuracy (1cm): 80.24 -> 77.62 | Hurt |
| use 3 refinement iterations instead of 2 (ablation) | Ours (LoFTR) · ETH3D | Accuracy (1cm): 80.24 -> 81.26 | Helped |
| use 2 views instead of 16 in multi-view matching (ablation) | Ours (LoFTR) · ETH3D | Accuracy (1cm): 77.62 -> 69.77 | Hurt |
| use 4 views instead of 16 in multi-view matching (ablation) | Ours (LoFTR) · ETH3D | Accuracy (1cm): 77.62 -> 72.30 | Hurt |
| use 8 views instead of 16 in multi-view matching (ablation) | Ours (LoFTR) · ETH3D | Accuracy (1cm): 77.62 -> 74.68 | Hurt |
| remove the multi-view feature transformer (ablation) | Ours (LoFTR) · ETH3D | Accuracy (1cm): 80.24 -> 75.12 | Hurt |
Surprising findings
- The coarse SfM model without refinement yields only 42.13% accuracy at 1cm on ETH3D, showing match quantization alone is insufficient for accuracy.
Limitations
- Detector-free matching and coarse SfM on 2000 images take 4.2 hours, making the framework slower than sparse-feature detector-based systems.
- Quantization sacrifices match accuracy in the coarse phase, which must be recovered by iterative refinement.
Experimental setup
Datasets: MegaDepth ETH3D IMC Phototourism 2021 Aachen v1.1
Benchmarks: ETH3D IMC 2021 Texture-Poor SfM dataset Aachen v1.1
Metrics: AUC of pose error at 1°, 3°, 5°, 10° accuracy and completeness at 1cm, 2cm, 5cm BA peak memory running time
Baselines: COLMAP (SIFT+NN) SIFT + NN + PixSfM D2Net + NN + PixSfM R2D2 + NN + PixSfM SP + SG + PixSfM LoFTR + PixSfM OnePose++
Builds on: LoFTR COLMAP PixSfM MatchFormer AspanTransformer
Closest papers by meaning
- VGGSfM: Visual Geometry Grounded Deep Structure From Motion (CVPR 2024 · similarity 0.88)
- Dense-SfM: Structure from Motion with Dense Consistent Matching (CVPR 2025 · similarity 0.88)
- Light3R-SfM: Towards Feed-forward Structure-from-Motion (CVPR 2025 · similarity 0.87)
- Leveraging Camera Triplets for Efficient and Accurate Structure-from-Motion (CVPR 2024 · similarity 0.86)
- RESfM: Robust Deep Equivariant Structure from Motion (ICLR 2025 · similarity 0.85)
- AnyMap: Learning a General Camera Model for Structure-from-Motion with Unknown Distortion in Dynamic Scenes (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.