Research map / Point clouds / Segmentation and detection
3DET-Mamba: Causal Sequence Modelling for End-to-End 3D Object Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes 3DET-Mamba, an end-to-end SSM-based indoor 3D object detector using Inner Mamba (local patches), Dual Mamba (global FPS/NPS token orders), and Query-aware Mamba decoder, outperforming 3DETR on ScanNet and SUN RGB-D.
Problem
Transformer-based 3D object detectors from point clouds have quadratic attention cost that limits resolution, and Mamba/SSMs are causal 1-D models that do not directly fit unordered 3D scene point clouds for detection.
Approach
A local-to-global Mamba scanning scheme: a lightweight Inner Mamba orders points within FPS/KNN patches by distance to capture local geometry, a Dual Mamba processes patch tokens in farthest-point and nearest-neighbor orders for global context, and a Query-aware Mamba decoder lets box queries attend to scene features via gated multiplicative interaction.
Key results
- On ScanNet, 3DET-Mamba reaches 66.9% mAP@0.25 and 48.7% mAP@0.5 vs 3DETR-m's 65.0%/47.0% with the same keypoints and queries.
- With 4096 points and 512 queries (3DET-Mamba†), ScanNet reaches 70.4% mAP@0.25 and 54.4% mAP@0.5.
- On SUN RGB-D, 3DET-Mamba† reaches 61.3% mAP@0.25 and 42.2% mAP@0.5, vs 59.1%/32.7% for 3DETR-m.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use 3DET-Mamba instead of 3DETR-m | 3D object detection · ScanNet val | mAP@0.25: 65.0 -> 66.9 | Helped |
| Use 3DET-Mamba instead of 3DETR-m | 3D object detection · ScanNet val | mAP@0.5: 47.0 -> 48.7 | Helped |
| Double the number of key points and queries | 3DET-Mamba · ScanNet val | mAP@0.5: +5.7 mAP@0.5 | Helped |
| Add Inner Mamba instead of using PointNet++ for local encoding (ablation) | 3DET-Mamba encoder with Transformer spatial encoding · ScanNet val | mAP@0.25: 63.1 -> 64.9 | Helped |
| Add Inner Mamba instead of using PointNet++ for local encoding (ablation) | 3DET-Mamba encoder with Dual Mamba · ScanNet val | mAP@0.5: 48.4 -> 48.7 | Helped |
| Add Dual Mamba instead of Transformer for spatial encoding (ablation) | 3DET-Mamba encoder with Inner Mamba · ScanNet val | mAP@0.5: 45.4 -> 48.7 | Helped |
| Use Dual Mamba instead of a single forward Mamba block (ablation) | 3DET-Mamba encoder · ScanNet val | mAP@0.5: +1.0% | Helped |
| Use Dual Mamba instead of forward-and-backward Bi-Mamba (ablation) | 3DET-Mamba encoder · ScanNet val | mAP@0.5: +1.2% | Helped |
| Use Query-aware Mamba instead of Transformer decoder (ablation) | 3DET-Mamba decoder · ScanNet val | mAP@0.5: 42.6 -> 48.7 | Helped |
| Use Query-aware Mamba instead of concatenating scene features and queries in an original Mamba decoder (ablation) | 3DET-Mamba decoder · ScanNet val | mAP@0.5: 28.0 -> 48.7 | Helped |
Surprising findings
- Replacing the Query-aware Mamba decoder with a naive Mamba decoder drops ScanNet mAP@0.5 to 28.0%, far below the Transformer decoder's 42.6%.
Limitations
- Only evaluated on indoor benchmarks (ScanNet, SUN RGB-D); meshes and other 3D data types not explored.
- Gains over 3DETR are modest at the matched setting (+1.9 mAP@0.25, +1.7 mAP@0.5 on ScanNet).
- No efficiency or runtime measurements are reported to support linear-complexity claims.
Experimental setup
Datasets: ScanNet V2 SUN RGB-D
Benchmarks: ScanNet V2 SUN RGB-D
Metrics: mAP@0.25 mAP@0.5
Baselines: 3DETR 3DETR-m VoteNet H3DNet BRNet GroupFree3D MLCVNet
Builds on: Mamba 3DETR PointMamba Vim PointNet++
Closest papers by meaning
- Voxel Mamba: Group-Free State Space Models for Point Cloud based 3D Object Detection (NeurIPS 2024 · similarity 0.89)
- UniMamba: Unified Spatial-Channel Representation Learning with Group-Efficient Mamba for LiDAR-based 3D Object Detection (CVPR 2025 · similarity 0.88)
- PointMamba: A Simple State Space Model for Point Cloud Analysis (NeurIPS 2024 · similarity 0.88)
- Mamba YOLO: A Simple Baseline for Object Detection with State Space Model (AAAI 2025 · similarity 0.88)
- State Space Model Meets Transformer: A New Paradigm for 3D Object Detection (ICLR 2025 · similarity 0.88)
- Spatial-Mamba: Effective Visual State Space Models via Structure-Aware State Fusion (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.