Research map / 3D vision / 3D generation
LowRankOcc: Tensor Decomposition and Low-Rank Recovery for Vision-based 3D Semantic Occupancy Prediction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LowRankOcc, which factorizes dense 3D occupancy voxel features into vertical vectors and horizontal matrices (VH decomposition with recursive residual decomposition) and recovers low-rank voxel tensors, avoiding 3D convolutions, for vision-based semantic occupancy prediction.
Problem
Dense voxel representations for vision-based 3D semantic occupancy prediction in outdoor driving scenes contain large spatial redundancy, which is costly in compute and memory and can lead to overfitting.
Approach
Exploit the low-rank structure of outdoor occupancy grids by decomposing the 3D tensor along the z-axis into vertical vector and horizontal matrix pairs (VH decomposition), applying recursive residual decomposition, encoding the components with 1D/2D convolutions in parallel mini-batches, and recovering a low-rank voxel tensor via outer products, instead of using 3D convolutions or 3D UNets.
Key results
- On SemanticKITTI test, LowRankOcc achieves 38.47 SC IoU and 13.56 SSC mIoU, beating SurroundOcc (34.72 / 11.86) among monocular methods.
- On nuScenes validation, LowRankOcc reaches 32.78 SC IoU and 21.51 SSC mIoU, improving over SurroundOcc (31.49 / 20.30) by about 1 point.
- Ablation on the VH rank shows performance rising up to R=4 (37.85 IoU, 14.21 mIoU) with diminishing gains beyond R=4, while using 8.4G memory versus 11.9G for a 3D UNet.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace the strongest reported baseline with LowRankOcc | SemanticKITTI semantic scene completion · SemanticKITTI test | SC IoU: 34.72 vs 38.47 | Helped |
| replace the strongest reported baseline with LowRankOcc | SemanticKITTI semantic scene completion · SemanticKITTI test | SSC mIoU: 12.32 vs 13.56 | Helped |
| replace the strongest reported baseline with LowRankOcc | nuScenes occupancy prediction · nuScenes validation | SC IoU: 31.49 vs 32.78 | Helped |
| replace the strongest reported baseline with LowRankOcc | nuScenes occupancy prediction · nuScenes validation | SSC mIoU: 20.30 vs 21.51 | Helped |
| add matrix encoding to vector encoding (ablation) | VH decomposition-based 3D encoding · SemanticKITTI validation | IoU: 30.01 -> 37.85 | Helped |
| replace VH decomposition with CP decomposition (ablation) | LowRankOcc 3D encoding · SemanticKITTI validation | IoU: 37.85 -> 31.10 | Hurt |
| replace VH decomposition with VM decomposition (ablation) | LowRankOcc 3D encoding · SemanticKITTI validation | IoU: 37.85 -> 37.69 | Hurt |
| set VH rank to 2 instead of 4 (ablation) | LowRankOcc · SemanticKITTI validation | IoU: 36.41 -> 37.85 | Hurt |
| set VH rank to 3 instead of 4 (ablation) | LowRankOcc · SemanticKITTI validation | IoU: 36.97 -> 37.85 | Hurt |
| increase VH rank from 4 to 5 (ablation) | LowRankOcc · SemanticKITTI validation | IoU: 37.85 -> 36.77 | Hurt |
| replace recursive with concurrent decomposition (ablation) | LowRankOcc at VH rank 4 · SemanticKITTI validation | IoU: 36.15 -> 37.85 | Hurt |
| add multi-scale feature interaction (ablation) | LowRankOcc · SemanticKITTI validation | IoU: 37.21 -> 37.85 | Helped |
Surprising findings
- In a toy experiment, the z-axis-only decomposition (TZ) outperformed the full VM decomposition in trade-off between accuracy and recovery speed.
Limitations
- Gains on nuScenes are modest (about 1 point over SurroundOcc).
- Performance gains saturate beyond VH rank R=4, requiring rank tuning.
- Some classes, such as small objects, remain weak on SemanticKITTI (e.g., person and bicyclist near 2%).
Experimental setup
Datasets: SemanticKITTI nuScenes SurroundOcc dense occupancy annotations
Benchmarks: SemanticKITTI nuScenes
Metrics: SC IoU SSC mIoU IoU mIoU
Baselines: MonoScene TPVFormer OccFormer SurroundOcc BEVFormer Atlas LMSCNet 3DSketch AICNet JS3C-Net
Builds on: CP decomposition vector-matrix decomposition OccFormer Mask2Former LSS
Closest papers by meaning
- OctOcc: High-Resolution 3D Occupancy Prediction with Octree (AAAI 2024 · similarity 0.92)
- OccMamba: Semantic Occupancy Prediction with State Space Models (CVPR 2025 · similarity 0.90)
- LOMA: Language-assisted Semantic Occupancy Network via Triplane Mamba (AAAI 2025 · similarity 0.90)
- ViPOcc: Leveraging Visual Priors from Vision Foundation Models for Single-View 3D Occupancy Prediction (AAAI 2025 · similarity 0.89)
- AdaOcc: Adaptive 3D Occupancy Prediction for Embodied Tasks (NeurIPS 2026 · similarity 0.89)
- OctreeOcc: Efficient and Multi-Granularity Occupancy Prediction Using Octree Queries (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.