Research map / 3D vision / 3D generation
Total-Decom: Decomposed 3D Scene Reconstruction with Minimal Interaction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Total-Decom decomposes neural implicit scene reconstructions into individual objects and background using SAM-distilled features on an extracted mesh plus a mesh-based region-growing step, requiring about one click per object.
Problem
Decomposed 3D scene reconstruction from multi-view posed images: separating a scene into object entities and background without dense human instance annotations, while keeping reconstruction quality and complete geometry.
Approach
Distill SAM encoder features into an implicit SDF-based neural field with foreground/background decomposition, extract a mesh with per-vertex SAM features, and grow object seeds from a single SAM-decoded click mask along mesh topology and feature similarity, bounded by mask boundaries.
Key results
- On Replica, scene reconstruction Chamfer-L1 3.53 vs 3.58 for ObjSDF++ and F-score 85.82 vs 85.69.
- Decomposed object reconstruction F-score 81.70 vs 79.49 for ObjSDF++ on Replica, using on average 1.41 clicks per object.
- Only method in the comparison table covering geometry, multi-grained, scene-level, single-view and interaction capabilities.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Total-Decom instead of ObjSDF++ | 3D scene reconstruction · Replica scene reconstruction | Chamfer-L1: 3.58 -> 3.53 | Helped |
| Use Total-Decom instead of ObjSDF++ | 3D scene reconstruction · Replica scene reconstruction | F-score: 85.69 -> 85.82 | Helped |
| Use Total-Decom instead of ObjSDF++ | 3D object decomposition · Replica decomposed reconstruction | Chamfer-L1: 3.84 ± 0.02 -> 3.58 ± 0.01 | Helped |
| Use Total-Decom instead of ObjSDF++ | 3D object decomposition · Replica decomposed reconstruction | F-score: 79.49 ± 0.08 -> 81.70 ± 0.08 | Helped |
| Add Manhattan regularization to the background reconstruction (ablation) | Total-Decom · Replica room 1 background reconstruction | background geometry regularity | Helped |
| Add floor reflection loss to the background reconstruction (ablation) | Total-Decom · Replica room 1 background reconstruction | background completeness | Helped |
| Distill SAM features instead of CLIP-LSeg features (ablation) | MonoSDF feature distillation · 3D object decomposition | object boundary preservation | Helped |
| Distill SAM features instead of DINO features (ablation) | MonoSDF feature distillation · 3D object decomposition | object boundary preservation | Helped |
Surprising findings
- Distilled SAM features in the feature field alone merge distant regions when grouped by feature similarity in 3D, so geometry and boundary constraints are necessary.
Limitations
- Decomposition on Replica is compared only against ObjSDF++ on the subset of objects ObjSDF++ can extract.
- Requires some human clicks (about 1.41 per object) rather than being fully automatic.
Experimental setup
Datasets: Replica ScanNet NICE-SLAM room self-captured billiard room
Benchmarks: Replica ScanNet
Metrics: Chamfer-L1 F-score
Baselines: ObjSDF++ ISRF DFF N3F SA3D
Builds on: MonoSDF ObjSDF++ VolSDF Segment Anything
Closest papers by meaning
- DM-NeRF: 3D Scene Geometry Decomposition and Manipulation from 2D Images (ICLR 2023 · similarity 0.88)
- Decompositional Neural Scene Reconstruction with Generative Diffusion Prior (CVPR 2025 · similarity 0.86)
- UniSDF: Unifying Neural Representations for High-Fidelity 3D Reconstruction of Complex Scenes with Reflections (NeurIPS 2024 · similarity 0.86)
- NTO3D: Neural Target Object 3D Reconstruction with Segment Anything (CVPR 2024 · similarity 0.86)
- Learning Partonomic 3D Reconstruction from Image Collections (CVPR 2025 · similarity 0.85)
- ANIM: Accurate Neural Implicit Model for Human Reconstruction from a single RGB-D Image (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.