Research map / 3D vision / Depth and stereo
Light3R-SfM: Towards Feed-forward Structure-from-Motion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Light3R-SfM performs feed-forward, optimization-free structure-from-motion by aligning DUSt3R/MASt3R-style pairwise pointmaps via a latent attention module and a shortest-path-tree scene graph.
Problem
Structure-from-Motion from unconstrained image collections normally needs costly image matching and global optimization (bundle adjustment or 3D global alignment), which is slow and memory-hungry for large collections.
Approach
Replace global optimization with a learnable latent global alignment module (self-attention over averaged global image tokens, then cross-attention to dense tokens) before pairwise pointmap decoding, combined with a shortest path tree scene graph of N-1 edges and Procrustes-based global accumulation of pointmaps.
Key results
- On Tanks&Temples with 200 views, Light3R-SfM reaches 52.4 RRA@5 and 53.1 RTA@5 at 33.4 s versus MASt3R-SfM at 1609 s, a >49x speedup on 200 images.
- On CO3Dv2 10-view setting, Light3R-SfM achieves mAA@30 of 72.8, the best among feed-forward methods, and it reaches 85.8 RTA@15 versus 80.9 for MASt3R*.
- On Waymo validation, Light3R-SfM attains 78.3 RRA@5 vs 75.7 for MASt3R-SfM at 8.5 s versus 1662 s runtime.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add latent alignment (ablation) | Light3R-SfM · Tanks&Temples 200 views | RRA@5: 47.5 -> 50.8 | Helped |
| add global supervision (ablation) | Light3R-SfM with latent alignment · Tanks&Temples 200 views | RRA@5: 50.8 -> 52.4 | Helped |
| replace MASt3R initialization with DUSt3R initialization (ablation) | Light3R-SfM · Tanks&Temples 200 views | RRA@5: 52.4 -> 48.8 | Hurt |
| replace SPT with MST graph construction (ablation) | Light3R-SfM · Tanks&Temples 200 views | RRA@5: 52.4 -> 44.4 | Hurt |
| replace SPT with Oracle graph construction (ablation) | Light3R-SfM · Tanks&Temples 200 views | RRA@5: 52.4 -> 52.8 | Helped |
| increase pointmap confidence threshold from 3 to 7 (ablation) | Light3R-SfM · Tanks&Temples 200 views | RRA@5: 56.5 -> 65.2 | Helped |
| use Light3R-SfM instead of MASt3R-SfM | SfM methods · Waymo validation split | Runtime: 1662.0 -> 8.5 | Helped |
| use Light3R-SfM instead of Spann3R | feed-forward SfM · Tanks&Temples 25 views | ATE: 0.060 -> 0.048 | Helped |
Surprising findings
- Spann3R in its offline unordered mode degrades strongly and runs out of memory on full sequences, while Light3R-SfM stays accurate.
- A minimum spanning tree scene graph yields large accuracy drops (about 15% RRA@5 and 26% RTA@5) compared to the shortest path tree.
Limitations
- Does not scale to collections of tens of thousands of images.
- Pose accuracy at tight thresholds lags behind optimization-based SfM, likely due to low input resolution.
- Drift in long camera trajectories and retrieval errors under extreme visual similarity.
Experimental setup
Datasets: Waymo Open Dataset CO3Dv2 MegaDepth TartanAir
Benchmarks: Tanks&Temples CO3Dv2 Waymo Open Dataset validation
Metrics: RRA@5 RTA@5 ATE registration rate runtime RRA@15 RTA@15 mAA@30
Baselines: COLMAP GLOMAP ACE0 DF-SfM FlowMap VGGSfM MASt3R-SfM Spann3R PoseDiff PoseReg RelPose++ DUSt3R-GA
Builds on: DUSt3R MASt3R VGGSfM Spann3R GLOMAP
Closest papers by meaning
- SPARS3R: Semantic Prior Alignment and Regularization for Sparse 3D Reconstruction (CVPR 2025 · similarity 0.88)
- VGGSfM: Visual Geometry Grounded Deep Structure From Motion (CVPR 2024 · similarity 0.88)
- Learning Structure-from-Motion with Graph Attention Networks (CVPR 2024 · similarity 0.87)
- MP-SfM: Monocular Surface Priors for Robust Structure-from-Motion (CVPR 2025 · similarity 0.87)
- Detector-Free Structure from Motion (CVPR 2024 · similarity 0.87)
- Leveraging Camera Triplets for Efficient and Accurate Structure-from-Motion (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.