Research map / Point clouds / Point cloud classification
SparseOcc: Rethinking Sparse Latent Representation for Vision-Based Semantic Occupancy Prediction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SparseOcc, a fully sparse (COO-format) occupancy network for vision-based semantic occupancy prediction, using a decomposed-kernel sparse latent diffuser, interpolation-based sparse feature pyramid, and sparse transformer head.
Problem
Dense 3D latent volumes in vision-based semantic occupancy prediction have cubic time and memory cost, while BEV/TPV projections lose geometric information.
Approach
Keep all 3D layers on a lossless sparse COO tensor: spatially decomposed sparse convolutions diffuse features into empty neighbors, interpolation-and-summation fuses multi-scale sparse features, and the Mask2Former-style transformer head decodes masks only over occupied voxels plus one shared empty token.
Key results
- On nuScenes-Occupancy, SparseOcc reaches 14.1% mIoU versus 12.8% for C-CONet while cutting FLOPs by 74.9% (455G vs 1810G).
- On SemanticKITTI validation, SparseOcc achieves 13.12 mIoU and 36.48 IoU with 393G FLOPs, about 44.2% of OccFormer's FLOPs.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace dense 3D representation with sparse latent representation and sparse operators | SparseOcc vs C-CONet · nuScenes-Occupancy val | mIoU: 12.8 -> 14.1 | Helped |
| replace dense 3D representation with sparse latent representation and sparse operators | SparseOcc vs C-CONet · nuScenes-Occupancy val | FLOPs: 1810G -> 455G | Helped |
| use sparse representation instead of TPV representation | SparseOcc vs TPVFormer · nuScenes-Occupancy val | mIoU: 7.8 -> 14.1 | Helped |
| use sparse representation instead of TPV representation | SparseOcc vs TPVFormer · nuScenes-Occupancy val | FLOPs: 1132G -> 455G | Helped |
| use SparseOcc instead of OccFormer | SparseOcc vs OccFormer · SemanticKITTI val | mIoU: 13.46 -> 13.12 | Hurt |
| use SparseOcc instead of OccFormer | SparseOcc vs OccFormer · SemanticKITTI val | FLOPs: 889G -> 393G | Helped |
| add one decomposed orthogonal sparse completion convolution instead of none (ablation) | SparseOcc · SemanticKITTI val | mIoU: 12.1 -> 13.1 | Helped |
| add regular 3×3×3 sparse completion convolutions instead of none (ablation) | SparseOcc · SemanticKITTI val | mIoU: 12.1 -> 12.2; 12.3; 12.6 | Helped |
| stack more than one decomposed sparse completion convolution (ablation) | SparseOcc · SemanticKITTI val | mIoU: 13.1 -> 12.8; 12.7 | Hurt |
| replace sparse interpolation decoder with MSDeformAttn3D (ablation) | SparseOcc · SemanticKITTI val | mIoU: 13.1 -> 13.3 | Helped |
| replace sparse interpolation decoder with FPN3D (ablation) | SparseOcc · SemanticKITTI val | mIoU: 13.1 -> 9.8 | Hurt |
| replace sparse transformer head with linear head (ablation) | SparseOcc · SemanticKITTI val | mIoU: 13.1 -> 11.8 | Hurt |
Surprising findings
- Even with no sparse completion block, SparseOcc performs satisfactorily, suggesting the sparse feature pyramid alone completes the scene to some extent.
- A linear head achieves the best geometry IoU (36.8) on SemanticKITTI, higher than the transformer heads, though it has lower mIoU.
Limitations
- IoU drops at higher input resolution (1600x900), attributed to hallucinations on empty voxels from over-dense sparse features.
- On SemanticKITTI, SparseOcc is only comparable to OccFormer rather than clearly superior in IoU and mIoU.
Experimental setup
Datasets: nuScenes-Occupancy SemanticKITTI
Benchmarks: nuScenes-Occupancy SemanticKITTI
Metrics: IoU mIoU FLOPs training GPU memory inference latency
Baselines: MonoScene TPVFormer OpenOccupancy C-CONet OccFormer LMSCNet 3DSketch AICNet JS3C-Net
Builds on: Mask2Former Lift-Splat-Shoot Cylinder3D spconv OccFormer
Closest papers by meaning
- OctOcc: High-Resolution 3D Occupancy Prediction with Octree (AAAI 2024 · similarity 0.90)
- COTR: Compact Occupancy TRansformer for Vision-based 3D Occupancy Prediction (CVPR 2024 · similarity 0.89)
- OPUS: Occupancy Prediction Using a Sparse Set (NeurIPS 2024 · similarity 0.89)
- LowRankOcc: Tensor Decomposition and Low-Rank Recovery for Vision-based 3D Semantic Occupancy Prediction (CVPR 2024 · similarity 0.89)
- SelfOcc: Self-Supervised Vision-Based 3D Occupancy Prediction (CVPR 2024 · similarity 0.88)
- ODG: Occupancy Prediction Using Dual Gaussians (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.