Research map / Point clouds / Segmentation and detection
FEAST-Mamba: FEAture and SpaTial Aware Mamba Network with Bidirectional Orthogonal Fusion for Cross-Modal Point Cloud Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FEAST-Mamba, a multi-modal (point, voxel, range image, RGB) point cloud segmentation network with bidirectional cross-modal attention, orthogonal fusion loss, and a feature-and-spatial dual-aware reordering for Mamba.
Problem
Multi-modal point cloud segmentation suffers from information loss or redundancy in fusion, and existing Mamba-based point cloud models order unordered points using only 3D spatial coordinates, ignoring feature similarity.
Approach
Combine bidirectional cross-modal attention between RGB, range-image and voxel features with an orthogonality loss on independent projections to reduce fused redundancy, and reorder point-cloud blocks for Mamba using nearest/next-nearest spatial neighbors plus global-feature cosine similarity.
Key results
- Achieves 73.2% mIoU on SemanticKITTI validation with RGB and range modalities, versus 72.3% for PTv3+PPT.
- Achieves 82.1% mAcc and 74.7% mIoU on S3DIS Area 5 and 77.8% mIoU on ScanNet (tied with ODIN).
- Ablation on SemanticKITTI: removing both reordering components drops mIoU from 73.2 to 71.2; proposed reordering beats Hilbert and Z-order curves by 0.9-1.1 points.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add feature- and spatial-aware reordering (ablation) | FEAST-Mamba · SemanticKITTI validation | mIoU: 71.2 -> 73.2 | Helped |
| Add feature-aware reordering (ablation) | FEAST-Mamba · SemanticKITTI validation | mIoU: 72.3 -> 73.2 | Helped |
| Add spatial-aware reordering (ablation) | FEAST-Mamba · SemanticKITTI validation | mIoU: 72.6 -> 73.2 | Helped |
| Add next-nearest reordering (ablation) | FEAST-Mamba · SemanticKITTI validation | mIoU: 73.0 -> 73.2 | Helped |
| Add BFE with orthogonal loss (ablation) | FEAST-Mamba · SemanticKITTI validation | mIoU: 72.4 -> 73.2 | Helped |
| Add bidirectional enhancement (ablation) | FEAST-Mamba · SemanticKITTI validation | mIoU: 72.7 -> 73.2 | Helped |
| Replace FEAST dual-aware reordering with random ordering (ablation) | FEAST-Mamba · SemanticKITTI validation | mIoU: 71.2 -> 73.2 | Hurt |
| Replace FEAST dual-aware reordering with serpentine curve ordering (ablation) | FEAST-Mamba · SemanticKITTI validation | mIoU: 71.9 -> 73.2 | Hurt |
| Replace FEAST dual-aware reordering with Hilbert curve ordering (ablation) | FEAST-Mamba · SemanticKITTI validation | mIoU: 72.3 -> 73.2 | Hurt |
| Replace FEAST dual-aware reordering with Z-order curve ordering (ablation) | FEAST-Mamba · SemanticKITTI validation | mIoU: 72.1 -> 73.2 | Hurt |
| Add RGB and Range inputs to Point input | FEAST-Mamba · SemanticKITTI validation | mIoU: 72.0 -> 73.2 | Helped |
Limitations
- Full multi-modal variant has 36.2M parameters and 15.2 FPS, much slower than the point-only variant.
- Relies on known LiDAR-camera calibration to establish cross-modal correspondences.
Experimental setup
Datasets: S3DIS ScanNet SemanticKITTI nuScenes
Benchmarks: S3DIS Area 5 ScanNet validation SemanticKITTI validation nuScenes
Metrics: OA mAcc mIoU parameters FLOPs FPS
Baselines: PTv3 PTv3+PPT PTv2 ODIN SPM RPVNet 2DPASS SphereFormer WaffleIron OA-CNNs
Builds on: Mamba PTv3 SPM RPVNet PMF
Closest papers by meaning
- Spectral Informed Mamba for Robust Point Cloud Processing (CVPR 2025 · similarity 0.86)
- Point Cloud Mamba: Point Cloud Learning via State Space Model (AAAI 2025 · similarity 0.86)
- Pamba: Enhancing Global Interaction in Point Clouds via State Space Model (AAAI 2025 · similarity 0.86)
- PMA: Towards Parameter-Efficient Point Cloud Understanding via Point Mamba Adapter (CVPR 2025 · similarity 0.85)
- 3DET-Mamba: Causal Sequence Modelling for End-to-End 3D Object Detection (NeurIPS 2024 · similarity 0.85)
- UniMamba: Unified Spatial-Channel Representation Learning with Group-Efficient Mamba for LiDAR-based 3D Object Detection (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.