Research map / 3D vision / 3D generation
STCOcc: Sparse Spatial-Temporal Cascade Renovation for 3D Occupancy and Scene Flow Prediction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes STCOcc, a sparse explicit occupancy-state-guided spatial-temporal cascade decoder for camera-based 3D occupancy and scene flow prediction that renovates voxel features with occlusion-aware attention and sparse long-term temporal fusion.
Problem
Vision-centric 3D occupancy and scene flow prediction relies on implicit learning to model spatial and temporal information, which struggles to capture local geometric detail and reduces spatial discriminability, while also being memory-heavy.
Approach
Explicitly use a self-recursive occupancy predictor's occupied state to reweight sampled features in spatial cross-attention (volume-rendering-inspired, combined with bin depth) and in temporal self-attention, and fuse long-term history sparsely by splitting non-empty (long-term) and empty (short-term) streams.
Key results
- Achieves 41.7% RayIoU and 44.6% mIoU on Occ3D-nuScenes at 704x256 with ResNet-50, using 7.7GB training memory.
- Reaches 40.8% RayIoU and 0.44 mAVE on OpenOcc, versus 33.4% RayIoU and 0.87 mAVE for SparseOcc, with 8.7GB memory.
- Ablation on OpenOcc shows the full combination raises RayIoU from 35.1% to 38.4% and lowers mAVE from 1.32 to 0.63.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add OA-SCA (ablation) | STCOcc baseline · OpenOcc ablation | RayIoU: 35.1 -> 35.7 | Helped |
| Add OA-SCA (ablation) | STCOcc baseline · OpenOcc ablation | mAVE: 1.32 -> 1.27 | Helped |
| Add SROP to OA-SCA (ablation) | STCOcc · OpenOcc ablation | RayIoU: 35.7 -> 36.0 | Helped |
| Add Sparse Temporal Fusion to OA-SCA and SROP (ablation) | STCOcc · OpenOcc ablation | RayIoU: 36.0 -> 38.0 | Helped |
| Add OA-TSA to OA-SCA, SROP, and STF (ablation) | STCOcc · OpenOcc ablation | RayIoU: 38.0 -> 38.4 | Helped |
| Use sparse rather than BEV temporal modeling (ablation) | STCOcc · OpenOcc ablation, 8 frames | RayIoU: 36.8 -> 37.5 | Helped |
| Use sparse rather than voxel temporal modeling (ablation) | STCOcc · OpenOcc ablation, 8 frames | RayIoU: 37.2 -> 37.5 | Helped |
| Use sparse rather than BEV temporal modeling (ablation) | STCOcc · OpenOcc ablation, 16 frames | RayIoU: 37.4 -> 38.4 | Helped |
| Replace vanilla TSA with OA-TSA (ablation) | STCOcc · OpenOcc ablation | RayIoU: 38.0 -> 38.3 | Helped |
| Replace vanilla SCA with OA-SCA (ablation) | STCOcc · OpenOcc ablation | RayIoU: 37.5 -> 38.3 | Helped |
| Replace DA-SCA with OA-SCA (ablation) | STCOcc · OpenOcc ablation | RayIoU: 37.7 -> 38.3 | Helped |
| Use STCOcc rather than COTR | ResNet-50 occupancy prediction · Occ3D-nus | RayIoU: 41.0 -> 41.7 | Helped |
Surprising findings
- Vanilla spatial cross-attention and depth-aware SCA (FB-BEV) give little or no improvement over the no-SCA baseline in the ablation (37.5 and 37.7 vs 37.6 RayIoU).
Limitations
- Spatial refinement depends on input image size, so higher resolution (1408x512) is needed for further gains, with more memory.
- Results on OpenOcc are compared against baselines with different backbones and input sizes, complicating fair comparison.
- Ablations are run on half of the training set.
Experimental setup
Datasets: Occ3D-nuScenes OpenOcc nuScenes
Benchmarks: Occ3D-nuScenes OpenOcc
Metrics: RayIoU mIoU mAVE GPU memory
Baselines: BEVFormer RenderOcc BEVDet-Occ FB-Occ SparseOcc COTR OPUS-L OccNet Let Occ Flow TPVFormer OSP ViewFormer
Builds on: BEVFormer FB-BEV SparseOcc BEVStereo MonoScene
Closest papers by meaning
- Rethinking Temporal Fusion with a Unified Gradient Descent View for 3D Semantic Occupancy Prediction (CVPR 2025 · similarity 0.89)
- VoxelSplat: Dynamic Gaussian Splatting as an Effective Loss for Occupancy and Flow Prediction (CVPR 2025 · similarity 0.89)
- LowRankOcc: Tensor Decomposition and Low-Rank Recovery for Vision-based 3D Semantic Occupancy Prediction (CVPR 2024 · similarity 0.88)
- ODG: Occupancy Prediction Using Dual Gaussians (NeurIPS 2025 · similarity 0.88)
- AdaOcc: Adaptive 3D Occupancy Prediction for Embodied Tasks (NeurIPS 2026 · similarity 0.88)
- 3D Occupancy Prediction with Low-Resolution Queries via Prototype-aware View Transformation (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.