Research map / 3D vision / 3D generation
LOMA: Language-assisted Semantic Occupancy Network via Triplane Mamba
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LOMA, a vision-language 3D semantic occupancy network that injects LSeg language features into voxel grids via a VL-aware scene generator and fuses vision and language features with a tri-plane Mamba block.
Problem
Vision-only 3D semantic occupancy prediction suffers from limited geometric information in images and restricted local feature interaction from modified attention in large outdoor scenes.
Approach
Use a pretrained vision-language model (LSeg) to lift pixel-level language features into 3D voxels as priors, and fuse vision and language 3D features with a tri-plane SSM (Mamba) block that projects onto XY, YZ, ZX planes for linear-complexity global interaction.
Key results
- On SemanticKITTI test, LOMA reaches 15.10 mIoU (best) and 43.01 IoU, vs. 15.04 mIoU for Symphonies.
- On SSCBench-KITTI360 test, LOMA reports 46.35 IoU and 18.28 mIoU, improving IoU by 2.23 over Symphonies.
- Ablation on SemanticKITTI val: full model reaches 44.23 IoU / 14.81 mIoU vs. 40.42 / 13.36 baseline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add LOMA's language-assisted VSG and TFM framework | Vision-based semantic occupancy prediction · SemanticKITTI test | mIoU: +0.06 points | Helped |
| Add LOMA's language-assisted VSG and TFM framework | Vision-based semantic occupancy prediction · SemanticKITTI test | IoU: +0.82 points | Helped |
| Add LOMA's language-assisted framework | Vision-based semantic occupancy prediction · SSCBench-KITTI360 test | IoU: 2.23 points | Helped |
| Add TFM to the baseline (ablation) | LOMA · SemanticKITTI validation | IoU: 40.42 -> 42.45 | Helped |
| Add TFM to the baseline (ablation) | LOMA · SemanticKITTI validation | mIoU: 13.36 -> 13.44 | Helped |
| Add VSG to the baseline (ablation) | LOMA · SemanticKITTI validation | IoU: 40.42 -> 41.47 | Helped |
| Add VSG to the baseline (ablation) | LOMA · SemanticKITTI validation | mIoU: 13.36 -> 13.45 | Helped |
| Add language information with VSG and TFM (ablation) | LOMA · SemanticKITTI validation | IoU: 1.16 IoU | Helped |
| Add language information with VSG and TFM (ablation) | LOMA · SemanticKITTI validation | mIoU: 0.88 mIoU | Helped |
| Use VL-language features instead of VL-vision features alone (ablation) | VSG · SemanticKITTI validation | mIoU: 14.14 -> 14.34 | Helped |
| Replace the SSM block with convolution (ablation) | LOMA TFM · SemanticKITTI validation | IoU: 44.23 -> 40.98 | Hurt |
| Replace the SSM block with deformable attention (ablation) | LOMA TFM · SemanticKITTI validation | IoU: 44.23 -> 43.70 | Hurt |
Limitations
- Relies on a pretrained depth network (MobileStereoNet) and a frozen VLM (LSeg), so performance depends on those components.
- Gains over Symphonies on SemanticKITTI are small (+0.06 mIoU).
- Multi-scale design with three scales increased parameters and FLOPs without benefit.
Experimental setup
Datasets: SemanticKITTI SSCBench-KITTI360
Benchmarks: SemanticKITTI SSCBench-KITTI360
Metrics: IoU mIoU
Baselines: MonoScene TPVFormer VoxFormer OccFormer SurroundOcc MonoOcc Symphonies HASSC
Builds on: Mamba MonoScene deformable attention LSeg
Closest papers by meaning
- OccMamba: Semantic Occupancy Prediction with State Space Models (CVPR 2025 · similarity 0.92)
- LowRankOcc: Tensor Decomposition and Low-Rank Recovery for Vision-based 3D Semantic Occupancy Prediction (CVPR 2024 · similarity 0.90)
- OctOcc: High-Resolution 3D Occupancy Prediction with Octree (AAAI 2024 · similarity 0.89)
- OctreeOcc: Efficient and Multi-Granularity Occupancy Prediction Using Octree Queries (NeurIPS 2024 · similarity 0.89)
- Rethinking Temporal Fusion with a Unified Gradient Descent View for 3D Semantic Occupancy Prediction (CVPR 2025 · similarity 0.88)
- RadOcc: Learning Cross-Modality Occupancy Knowledge through Rendering Assisted Distillation (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.