Research map / 3D vision / 3D generation

SDGOCC: Semantic and Depth-Guided Bird's-Eye View Transformation for 3D Multimodal Occupancy Prediction

CVPR 2025 · 3D vision › 3D generation · approach: Vision-based 3D occupancy prediction · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SDG-OCC, a LiDAR-camera 3D occupancy prediction network with a semantic- and depth-guided BEV view transformation and fusion-to-occupancy-driven active distillation, achieving SOTA mIoU on Occ3D-nuScenes at real-time speed.

Problem

Camera-only occupancy methods lack accurate depth, LiDAR-only methods suffer from occlusion, and LSS-based lightweight pipelines produce sparse, inaccurately estimated BEV features while underusing LiDAR geometry and semantics.

Approach

Refine LSS depth distributions using LiDAR co-point depth diffused within same-semantic regions (from a segmentation head) followed by bilinear discretization, and fuse LiDAR and camera BEV features with neighborhood gated attention, then selectively distill fused knowledge into the camera branch only in LiDAR-occupied regions.

Key results

• SDG-Fusion reaches 51.66 mIoU on Occ3D-nuScenes validation (133 ms), above RadOcc-LC's 49.38 and the camera-only COTR's 46.2.
• SDG-KL reaches 50.16 mIoU on Occ3D-nuScenes at 83 ms, trading a small accuracy drop for real-time speed.
• On SurroundOcc-nuScenes validation, SDG-Fusion reaches 31.7 mIoU, above DAOcc's 30.5 under the standard protocol; visible-mask variants reach 52.2 mIoU.

Measured effects reported in the paper

Limitations

• Performance on SurroundOcc-nuScenes is only comparable, not clearly superior, to the best prior methods under the standard protocol.
• Best SurroundOcc numbers rely on visible masks generated from a prior method, which is a different protocol.
• Hyperparameters (radius, number of diffusion layers, K, alpha/beta) were tuned on the validation set.

Experimental setup

Datasets: Occ3D-nuScenes SurroundOcc-nuScenes

Benchmarks: Occ3D-nuScenes SurroundOcc-nuScenes

Metrics: mIoU IoU per-class IoU FPS inference time (ms)

Baselines: TPVFormer SurroundOcc OccFormer FBOcc PanoOcc FastOcc BEVDet4D FlashOcc COTR HyDRa OCCFusion RadOcc-LC

Builds on: Lift-Splat-Shoot FlashOcc BEVFusion SPVCNN FOAD active distillation idea from prior work

Closest papers by meaning

• Achieving Speed-Accuracy Balance in Vision-based 3D Occupancy Prediction via Geometric-Semantic Disentanglement (AAAI 2025 · similarity 0.90)
• CSV-Occ: Fusing Multi-frame Alignment for Occupancy Prediction with Temporal Cross State Space Model and Central Voting Mechanism (ICML 2025 · similarity 0.90)
• Spatiotemporal Decoupling for Efficient Vision-Based Occupancy Forecasting (CVPR 2025 · similarity 0.89)
• UnO: Unsupervised Occupancy Fields for Perception and Forecasting (CVPR 2024 · similarity 0.89)
• SelfOcc: Self-Supervised Vision-Based 3D Occupancy Prediction (CVPR 2024 · similarity 0.89)
• OctOcc: High-Resolution 3D Occupancy Prediction with Octree (AAAI 2024 · similarity 0.89)