Research map / Detection and segmentation / 3D detection and driving
CSV-Occ: Fusing Multi-frame Alignment for Occupancy Prediction with Temporal Cross State Space Model and Central Voting Mechanism
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CSV-Occ, a camera-based 3D semantic occupancy predictor that fuses multi-frame voxel features with a cascaded Cross State Space Module and refines instance interiors via a Voting-based Enhancement Mechanism.
Problem
Camera-based 3D semantic occupancy prediction suffers from quadratic cost of attention-based temporal fusion and from internal occupancy vacancy (object centers predicted as empty voxels).
Approach
Extends the state space model to take multiple input sequences (current and historical voxel features, with C mapped from the current frame) in a cascaded temporal fusion, achieving linear complexity; adds relative central regression that lets foreground voxels vote for their instance center and a parallel interaction strategy that propagates features from the center to the voting voxels.
Key results
- CSV-Occ-B (4f) achieves 44.93 mIoU on Occ3D-nuScenes validation, above PanoOcc (4f, R101) at 42.13 and OctreeOcc (4f) at 44.02.
- CSV-Occ-S (4f, R50) reaches 42.37 mIoU on Occ3D-nuScenes, above FB-OCC (8f, R50) at 40.67.
- CSV-Occ-B (4f) reaches 73.4 mIoU on nuScenes LiDAR segmentation via projection, the best among image-based methods listed (vs PanoOcc R101 71.6).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CSV-Occ-B instead of OctreeOcc (4F) | Occ3D-nuScenes occupancy prediction · Occ3D-nuScenes val | mIoU: 44.02 -> 44.93 | Helped |
| Use CSV-Occ-B instead of PanoOcc (4F) | camera-based LiDAR semantic segmentation · nuScenes val | mIoU: 71.6 -> 73.4 | Helped |
| Use Cross SSM with state dimension 4 instead of temporal self-attention, MLP-Mixer, or concatenation and convolution (ablation) | CSV-Occ-B multi-frame voxel feature aggregation · Occ3D-nuScenes val | mIoU (Occ.): 43.17 -> 44.93; 43.22 -> 44.93; 42.31 -> 44.93 | Helped |
| Set Cross SSM state dimension to 2 instead of 4 (ablation) | CSV-Occ-B multi-frame voxel feature aggregation · Occ3D-nuScenes val | mIoU (Occ.): 44.93 -> 44.77 | Hurt |
| Set Cross SSM state dimension to 8 instead of 4 (ablation) | CSV-Occ-B multi-frame voxel feature aggregation · Occ3D-nuScenes val | mIoU (Occ.): 44.93 -> 44.89 | Hurt |
| Set Cross SSM state dimension to 16 instead of 4 (ablation) | CSV-Occ-B multi-frame voxel feature aggregation · Occ3D-nuScenes val | mIoU (Occ.): 44.93 -> 43.58 | Hurt |
| Add TVT multi-frame fusion (ablation) | CSV-Occ-B · Occ3D-nuScenes val | mIoU (Occ.): 40.37 -> 43.71 | Helped |
| Add VEM (ablation) | CSV-Occ-B with TVT · Occ3D-nuScenes val | mIoU (Occ.): 43.71 -> 44.93 | Helped |
| Add TVT multi-frame fusion (ablation) | CSV-Occ-B · nuScenes val | mIoU (Seg.): 69.81 -> 72.77 | Helped |
| Add VEM (ablation) | CSV-Occ-B with TVT · nuScenes val | mIoU (Seg.): 72.77 -> 73.42 | Helped |
| Add RCR loss with weight 1 (ablation) | CSV-Occ-B loss · Occ3D-nuScenes val | mIoU (Occ.): 40.61 -> 42.83 | Helped |
| Set RCR loss weight to 2 instead of 1 (ablation) | CSV-Occ-B loss · Occ3D-nuScenes val | mIoU (Occ.): 44.93 -> 43.98 | Hurt |
Surprising findings
- Directly adding or concatenating multi-frame voxel features degrades performance relative to the cascaded Cross SSM design, per the authors' ablation.
- Increasing input frames from 4 to 8 can make internal occupancy vacancy worse, so more frames do not fix the IOV problem.
Limitations
- Gains over the strongest prior image-based method are modest (e.g., 44.93 vs 44.02 for OctreeOcc), with no variance or multi-seed reporting.
- Voting only applies to the 10 foreground classes, and the approach only refines instance interiors.
Experimental setup
Datasets: Occ3D-nuScenes nuScenes
Benchmarks: Occ3D-nuScenes nuScenes LiDAR semantic segmentation
Metrics: mIoU samples per second inference memory trainable params
Baselines: MonoScene BEVDet OccFormer TPVFormer CTF-Occ BEVFormer BEVDet (8f) PanoOcc FB-OCC OctreeOcc RangeNet++ PolarNet
Builds on: Mamba BEVFormer FB-OCC deformable attention scene-class affinity loss
Closest papers by meaning
- Achieving Speed-Accuracy Balance in Vision-based 3D Occupancy Prediction via Geometric-Semantic Disentanglement (AAAI 2025 · similarity 0.92)
- OctOcc: High-Resolution 3D Occupancy Prediction with Octree (AAAI 2024 · similarity 0.91)
- SDGOCC: Semantic and Depth-Guided Bird's-Eye View Transformation for 3D Multimodal Occupancy Prediction (CVPR 2025 · similarity 0.90)
- ViPOcc: Leveraging Visual Priors from Vision Foundation Models for Single-View 3D Occupancy Prediction (AAAI 2025 · similarity 0.90)
- Collaborative Semantic Occupancy Prediction with Hybrid Feature Fusion in Connected Automated Vehicles (CVPR 2024 · similarity 0.90)
- ODG: Occupancy Prediction Using Dual Gaussians (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.