Research map / 3D vision / 3D generation
Living Scenes: Multi-object Relocalization and Reconstruction in Changing 3D Environments
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MORE2, an SE(3)-equivariant vector-neuron encoder-decoder with DeepSDF decoder, trained on synthetic ShapeNet-based living scenes, that jointly handles instance matching, registration, and reconstruction across multi-temporal scans, with a joint shape-pose optimization for accumulation.
Problem
Long-term changes in indoor 3D environments observed through sparse, irregular scans require multi-object relocalization (matching and 6DoF registration) and reconstruction of object instances that improves over time.
Approach
A single category-agnostic SIM(3)-equivariant vector-neuron encoder with a DeepSDF decoder provides embeddings used for matching (Hungarian on invariant and equivariant similarity), SDF-based registration on SE(3), reconstruction, and a joint shape-pose optimization that accumulates point clouds across scans.
Key results
- End-to-end MRR recall on FlyingShape rises from 54.30 (baseline) to 62.00, and on 3RScan from 30.77 to 40.74.
- Instance matching on FlyingShape reaches 88.75 instance-level recall vs 86.63 for the best baseline VN-ONet.
- Registration recall on FlyingShape improves to 83.83 vs 77.67 for GeoTransformer, and on 3RScan to 61.11 vs 51.71.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MORE2 instead of the combined baseline | End-to-end relocalization and reconstruction · FlyingShape | MRR recall: 54.30 -> 62.00 | Helped |
| Use MORE2 instead of the combined baseline | End-to-end relocalization and reconstruction · 3RScan | MRR recall: 30.77 -> 40.74 | Helped |
| Use MORE2 instead of VN-ONet recon | Instance matching · FlyingShape | instance-level matching recall: 86.63 -> 88.75 | Helped |
| Use MORE2 instead of VN-ONet recon | Instance matching · 3RScan | instance-level matching recall (All): 64.83 -> 71.77 | Helped |
| Add registration optimization (ablation) | MORE2 · FlyingShape | registration recall: 83.00 -> 83.83 | Helped |
| Add registration optimization (ablation) | MORE2 · 3RScan | RMSE: 5.49 -> 4.74 | Helped |
| Add registration optimization (ablation) | MORE2 · 3RScan | median Chamfer Distance: 0.0032 -> 0.0030 | Helped |
| Add joint shape-pose optimization (ablation) | MORE2 · FlyingShape | IoU: 49.98 -> 66.73 | Helped |
| Add joint shape-pose optimization (ablation) | MORE2 · 3RScan | L1-Chamfer1: 9.28 -> 6.16 | Helped |
| Add joint shape-pose optimization (ablation) | MORE2 · 3RScan | SDF recall: 56.87 -> 64.12 | Helped |
| Accumulate point clouds from additional temporal scans (ablation) | MORE2 · FlyingShape | registration rotation error and reconstruction Chamfer distance | Helped |
Surprising findings
- Joint optimization with accumulation shows coverage saturating after about four scans, so additional scans add little to registration and reconstruction.
Limitations
- Does not handle elastic deformations of objects
- Struggles with multiple identical objects in the scene
- Evaluated on a limited 3RScan validation subset with incomplete, non-watertight object meshes
Experimental setup
Datasets: ShapeNet FlyingShape 3RScan
Benchmarks: FlyingShape 3RScan
Metrics: matching recall registration recall MedRE RMSE MedCD Chamfer distance volumetric IoU SDF recall MR recall MRR recall
Baselines: MendNet VN-ONet VN-DGCNN ConvONet RPMNet FreeReg GeoTransformer
Builds on: Vector Neurons DeepSDF SIM(3)-equivariant shape embedding (EFEM) VN-Transformer Kabsch algorithm
Closest papers by meaning
- Multi-View Pose-Agnostic Change Localization with Zero Labels (CVPR 2025 · similarity 0.87)
- Adaptive 3D Reconstruction via Diffusion Priors and Forward Curvature-Matching Likelihood Updates (NeurIPS 2025 · similarity 0.87)
- Continuous 3D Perception Model with Persistent State (CVPR 2025 · similarity 0.87)
- BAT3R: Robust Online 3D Reconstruction with Bayesian Adaptive State Updates (NeurIPS 2026 · similarity 0.86)
- Decompositional Neural Scene Reconstruction with Generative Diffusion Prior (CVPR 2025 · similarity 0.86)
- ReMatching Dynamic Reconstruction Flow (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.