Research map / 3D vision / 3D generation
Point3R: Streaming 3D Reconstruction with Explicit Spatial Pointer Memory
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Point3R, a streaming 3D reconstruction model that keeps an explicit spatial pointer memory (3D-positioned pointers with features), updated via pointer-image interaction with 3D hierarchical RoPE and a fusion mechanism.
Problem
Dense streaming 3D reconstruction from ordered or unordered image collections, where implicit fixed-size or growing feature memories suffer from capacity limits, redundancy, and information loss of earlier frames.
Approach
Replace implicit state/token memory with an explicit memory of 3D-positioned pointers, each carrying a spatial feature aggregated from nearby observations; interactions use a 3D hierarchical rotary position embedding over pointer and image-token positions, and nearby new pointers are fused into existing ones.
Key results
- On 7-scenes sparse inputs, Point3R reaches mean Acc 0.085 vs CUT3R 0.126 and DUSt3R-GA 0.146 (Table 1).
- On long sequences, Point3R reduces 7-scenes mean Acc from 0.238 (CUT3R) to 0.071.
- Monocular depth on Bonn gives Abs Rel 0.061, best among methods listed; ScanNet pose ATE is 0.097 vs CUT3R 0.099 (comparable).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Point3R instead of CUT3R | streaming 3D reconstruction · 7-scenes (long sequences) | Acc (Mean): 0.238 -> 0.071 | Helped |
| Use Point3R instead of CUT3R | streaming 3D reconstruction · NRGBD (long sequences) | NC (Mean): 0.556 -> 0.641 | Helped |
| Add 3D hierarchical position embedding (ablation) | Point3R · 7-scenes | NC: 0.698 -> 0.725 | Helped |
| Add 3D hierarchical position embedding (ablation) | Point3R · NRGBD | Comp: 0.079 -> 0.073 | Helped |
| Add memory fusion (ablation) | Point3R · 7-scenes | Comp: 0.148 -> 0.139 | Helped |
| Add memory fusion (ablation) | Point3R · 7-scenes | Acc: 0.118 -> 0.124 | Hurt |
| Shuffle the input sequence rather than keep it ordered | Point3R · 7-scenes | Acc (Mean): 0.032 -> 0.033 | No clear effect |
| Shuffle the input sequence rather than keep it ordered | Point3R · NRGBD | NC (Mean): 0.801 -> 0.800 | No clear effect |
| Use Point3R instead of CUT3R | monocular depth estimation · NYU-v2 (Static) | Abs Rel: 0.086 -> 0.078 | Helped |
| Use Point3R instead of MASt3R | monocular depth estimation · Sintel | Abs Rel: 0.340 -> 0.395 | Hurt |
| Use Point3R instead of CUT3R | video depth estimation with metric-scale alignment · Bonn | Abs Rel: 0.103 -> 0.081 | Helped |
| Use Point3R instead of CUT3R | camera pose estimation · ScanNet (Static) | ATE: 0.099 -> 0.097 | No clear effect |
Surprising findings
- Pose estimation is not state-of-the-art: on Sintel and ScanNet, Point3R's RPE rot is markedly worse than CUT3R and optimization-based baselines (e.g., 2.791 vs 0.600 on ScanNet).
- Memory fusion slightly degrades some reconstruction metrics (e.g., 7-scenes Acc 0.118 without fusion vs 0.124 with), trading accuracy for efficiency.
Limitations
- As explored area grows, stored pointers may interfere with camera pose estimation for later frames.
- Pose estimation accuracy lags optimization-based methods and some online baselines on Sintel and ScanNet.
- Per-frame runtime is higher than MASt3R-SLAM.
Experimental setup
Datasets: ARKitScenes ScanNet ScanNet++ CO3Dv2 WildRGBD OmniObject3D HyperSim BlendedMVS MegaDepth Waymo VirtualKITTI2 PointOdyssey
Benchmarks: 7-scenes NRGBD NYU-v2 Sintel Bonn KITTI ScanNet TUM-dynamics
Metrics: Acc Comp NC Abs Rel delta<1.25 ATE RPE trans RPE rot
Baselines: DUSt3R-GA MASt3R-GA MonST3R-GA Spann3R CUT3R MASt3R-SLAM
Builds on: DUSt3R CUT3R RoPE MASt3R
Closest papers by meaning
- SLAM3R: Real-Time Dense Scene Reconstruction from Monocular RGB Videos (CVPR 2025 · similarity 0.88)
- Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Pass (CVPR 2025 · similarity 0.87)
- Continuous 3D Perception Model with Persistent State (CVPR 2025 · similarity 0.87)
- Test3R: Learning to Reconstruct 3D at Test Time (NeurIPS 2025 · similarity 0.87)
- SPAR3D: Stable Point-Aware Reconstruction of 3D Objects from Single Images (CVPR 2025 · similarity 0.87)
- Enhancing 3D Reconstruction for Dynamic Scenes (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.