Research map / 3D vision / Depth and stereo
Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Pass
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Fast3R extends DUSt3R into a transformer that jointly processes many unposed views in one forward pass to predict pointmaps, avoiding pairwise inference and global alignment.
Problem
DUSt3R-style 3D reconstruction is pairwise: it needs O(N^2) pairwise pointmaps and a costly global alignment, which is slow, memory-hungry, and limits context to pairs.
Approach
A multi-view transformer with all-to-all attention over concatenated patch tokens from all views, with per-view index embeddings sampled randomly from a larger pool (position-interpolation style) so it can generalize from ~20 training views to 1000+ test views, predicting local and global pointmaps in a single pass.
Key results
- On CO3Dv2 Fast3R-no-outdoor reaches 99.7 RRA@15, versus 96.2 for DUSt3R, with about 320x faster inference (251.1 FPS vs 0.78).
- Fast3R processes up to 1500 views in one forward pass on a single A100, while DUSt3R runs out of memory beyond 32 views.
- Accuracy and completion improve as the number of inference views grows, and models trained on more views improve on reconstruction and pose metrics.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Fast3R instead of MASt3R | multi-view pose regression · CO3Dv2 | RRA@15°: 94.6 -> 96.2 | Helped |
| use Fast3R instead of MASt3R | multi-view pose regression · CO3Dv2 | RTA@15°: 91.9 -> 81.6 | Hurt |
| use Fast3R instead of DUSt3R | multi-view pose regression · CO3Dv2 | FPS: 0.78 -> 251.1 | Helped |
| use Fast3R instead of Spann3R | 3D reconstruction · 7-Scenes | Acc#: 1.48 -> 1.58 | Hurt |
| use Fast3R instead of Spann3R | 3D reconstruction · NRGBD | Comp#: 1.10 -> 1.01 | Helped |
| use the local pointmap aligned to global instead of the global pointmap directly (ablation) | Fast3R · 7-Scenes | Acc#: 4.81 -> 2.84 | Helped |
| use the local pointmap aligned to global instead of the global pointmap directly (ablation) | Fast3R · 7-Scenes | Comp#: 1.64 -> 1.37 | Helped |
| use the local pointmap aligned to global instead of the global pointmap directly (ablation) | Fast3R · NRGBD | Acc#: 4.85 -> 4.39 | Helped |
| use the local pointmap aligned to global instead of the global pointmap directly (ablation) | Fast3R · DTU | Acc#: 3.88 -> 3.91 | No clear effect |
| train with randomized image-index position embeddings (ablation) | Fast3R · inference with more views than seen during training | pointmap regression loss | Helped |
| train with more views per sample (ablation) | Fast3R · CO3D camera pose estimation | RRA and RTA | Helped |
| train with more views per sample (ablation) | Fast3R · 7-Scenes and NRGBD reconstruction | Acc# and Comp# | Helped |
Surprising findings
- Fast3R's local pointmap head, aligned to the global map with ICP, outperforms the global head on 7-Scenes and NRGBD reconstruction accuracy.
- Models evaluated on more views than seen during training (e.g., 50 vs 20) improve per-view accuracy.
Limitations
- Drift in pointmaps for very large scenes when the number of views exceeds about 300, particularly for low-confidence views.
- Scaling is limited by data accuracy and quantity; training uses real scans with systematic label errors.
- Removing BlendedMVS and MegaDepth improves CO3D pose accuracy but hurts generalization to diverse outdoor scenes.
Experimental setup
Datasets: CO3D ScanNet++ ARKitScenes Habitat BlendedMVS MegaDepth CO3Dv2 RealEstate10K 7-Scenes NRGBD DTU
Benchmarks: CO3Dv2 RealEstate10K 7-Scenes NRGBD DTU
Metrics: RRA@15 RTA@15 RRA@5 RTA@5 mAA(30) FPS Accuracy (median distance) Completion (median distance) peak GPU memory
Baselines: DUSt3R MASt3R Spann3R Colmap+SG PixSfM RelPose PoseDiff RelPose++ F-Recon
Builds on: DUSt3R CroCo ViT DPT Position Interpolation FlashAttention
Closest papers by meaning
- DUSt3R: Geometric 3D Vision Made Easy (CVPR 2024 · similarity 0.93)
- MUSt3R: Multi-view Network for Stereo 3D Reconstruction (CVPR 2025 · similarity 0.92)
- MV-DUSt3R+: Single-Stage Scene Reconstruction from Sparse Views In 2 Seconds (CVPR 2025 · similarity 0.91)
- Pow3R: Empowering Unconstrained 3D Reconstruction with Camera and Scene Priors (CVPR 2025 · similarity 0.88)
- Test3R: Learning to Reconstruct 3D at Test Time (NeurIPS 2025 · similarity 0.88)
- Point3R: Streaming 3D Reconstruction with Explicit Spatial Pointer Memory (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.