Research map / Point clouds / Point cloud classification
DOCTR: Disentangled Object-Centric Transformer for Point Scene Understanding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DOCTR, an object-centric Transformer decoder with semantic-geometry disentangled queries and hybrid bipartite matching that jointly segments, detects, and reconstructs object meshes from scene point clouds on ScanNet.
Problem
Point scene understanding requires jointly segmenting objects, estimating their poses, and reconstructing their meshes from noisy real-world scene point clouds; existing multi-stage pipelines (DIMR, RfD-Net) process objects independently and struggle with nearby objects.
Approach
Represent each object as a query in a Transformer decoder and split each query into a semantic part (supervised by mask and class) and a geometric part (supervised by box and shape), with separate cross-attention for each part, so that all objects and sub-tasks are optimized jointly in a compact, NMS-free pipeline.
Key results
- DOCTR w/ MEBR reaches IoU@0.25 of 58.25 and CD@0.1 of 59.61 on ScanNet, versus 46.34 and 52.39 for DIMR (original report).
- DOCTR w/ MEBR surpasses DIMR (refined data) on all reported metrics including PCR@0.5 (59.89 vs 56.76).
- Object recognition Prec@0.25 rises to 49.14 from 36.23 for DIMR (refined data), without NMS.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add mask-enhanced box refinement (MEBR) (ablation) | DOCTR without MEBR · ScanNet | IoU@0.25: 55.18 -> 58.25 | Helped |
| add mask-enhanced box refinement (MEBR) (ablation) | DOCTR without MEBR · ScanNet | LFD@2500: 12.00 -> 10.9 | Hurt |
| add mask-enhanced box refinement (MEBR) (ablation) | DOCTR without MEBR · ScanNet | PCR@0.5: 55.70 -> 59.89 | Helped |
| add mask-enhanced box refinement (MEBR) (ablation) | DOCTR without MEBR · ScanNet | Prec.@0.25: 47.75 -> 49.14 | Helped |
| add mask-enhanced box refinement (MEBR) (ablation) | DOCTR without MEBR · ScanNet | Prec.@0.5: 21.08 -> 21.63 | Helped |
| add SGDQ (ablation) | Mask3D-based baseline with box and shape heads · ScanNet | IoU@0.25: 52.59 -> 53.93 | Helped |
| add SGDQ (ablation) | Mask3D-based baseline with box and shape heads · ScanNet | CD@0.1: 52.51 -> 54.79 | Helped |
| add hybrid bipartite matching (ablation) | Mask3D-based baseline with box and shape heads · ScanNet | IoU@0.25: 52.59 -> 52.51 | Hurt |
| add hybrid bipartite matching (ablation) | Mask3D-based baseline with box and shape heads · ScanNet | CD@0.1: 52.51 -> 53.31 | Helped |
| add SGDQ and hybrid bipartite matching (ablation) | Mask3D-based baseline with box and shape heads · ScanNet | IoU@0.25: 52.59 -> 55.18 | Helped |
| add SGDQ and hybrid bipartite matching (ablation) | Mask3D-based baseline with box and shape heads · ScanNet | CD@0.1: 52.51 -> 56.82 | Helped |
Surprising findings
- Straightforwardly adding box and shape heads to a Mask3D-style object-centric Transformer underperforms DIMR, and hybrid matching alone slightly lowers IoU@0.5.
Limitations
- Relies on a pretrained, frozen shape decoder (BSP-Net) rather than end-to-end mesh generation.
- PCR@0.5 is lower than DIMR without MEBR, attributed to sensitivity to box deviation.
Experimental setup
Datasets: ScanNet V2 Scan2CAD ShapeNet
Benchmarks: ScanNet V2
Metrics: IoU@0.25 IoU@0.5 CD@0.1 CD@0.047 LFD@5000 LFD@2500 PCR@0.5 Prec@0.25 Prec@0.5
Baselines: RfD-Net DIMR Mask3D-based baseline
Builds on: DETR Mask3D DIMR RfD-Net BSP-Net
Closest papers by meaning
- ImageNet3D: Towards General-Purpose Object-Level 3D Understanding (NeurIPS 2024 · similarity 0.87)
- MiKASA: Multi-Key-Anchor & Scene-Aware Transformer for 3D Visual Grounding (CVPR 2024 · similarity 0.86)
- AGILE3D: Attention Guided Interactive Multi-object 3D Segmentation (ICLR 2024 · similarity 0.86)
- A Unified Framework for 3D Scene Understanding (NeurIPS 2024 · similarity 0.86)
- OneFormer3D: One Transformer for Unified Point Cloud Segmentation (CVPR 2024 · similarity 0.86)
- Point-based Instance Completion with Scene Constraints (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.