Research map / 3D vision / 3D generation

Skip Mamba Diffusion for Monocular 3D Semantic Scene Completion

AAAI 2025 · 3D vision › 3D generation · approach: Semantic scene completion · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SkimbaDiff, a monocular 3D semantic scene completion model that runs a Skimba (skip triple Mamba) diffusion denoiser in a VAE latent space, with a Skimba-based segmentation sub-network, and reports state-of-the-art monocular results on SemanticKITTI and SSCBench-KITTI-360.

Problem

Monocular 3D semantic scene completion: inferring missing geometry and semantics for every voxel from a single RGB image, where existing monocular methods lag behind LiDAR and stereo methods.

Approach

Combines latent diffusion in a VAE latent space with a Mamba-based denoiser (Skimba) that uses skip triple Mamba layers with dilations 0, 1, and 3 across forward, reverse, and spatial scan directions, plus dimensional decomposition residual blocks in a semantic block for 3D context.

Key results

• On the SemanticKITTI test set, SkimbaDiff reaches 46.95% IoU and 13.78% mIoU with monocular input, the best among monocular methods and above several stereo methods.
• On SSCBench-KITTI-360 test, SkimbaDiff achieves 41.92% IoU and 14.40% mIoU, outperforming monocular baselines such as OccFormer (40.27 / 13.81) and surpassing the stereo DepthSSC mIoU (14.28).
• Ablation on SemanticKITTI validation: the full model reaches 43.82 IoU, versus 34.91 without MSCB, 39.87 without SB, and 37.58 without Skimba.

Measured effects reported in the paper

Surprising findings

• Inference is slow: the Skimba denoiser alone takes 7.38 s of an 8.13 s full-model inference time, using 100 denoising steps.

Limitations

• Diffusion denoising with 100 steps makes inference slow (about 8 s per scene), which the authors frame as acceptable for offline use only.
• Performance on SSCBench-KITTI-360 still trails LiDAR-based methods, which the authors attribute to dataset quality.
• Several small-object classes (bicycle, motorcyclist, person) score near zero or low IoU.

Experimental setup

Datasets: SemanticKITTI SSCBench-KITTI-360

Benchmarks: SemanticKITTI SSCBench-KITTI-360

Metrics: IoU mIoU

Baselines: LMSCNet AICNet JS3C-Net MonoScene TPVFormer SurroundOcc OccFormer IAMSSC VoxFormer DepthSSC HASSC-S H2GFormer-S

Builds on: latent diffusion Mamba DDR SCPNet Lovasz-softmax loss

Closest papers by meaning

• SemCity: Semantic Scene Generation with Triplane Diffusion (CVPR 2024 · similarity 0.87)
• Scaling Diffusion Mamba with Bidirectional SSMs for Efficient 3D Shape Generation (AAAI 2025 · similarity 0.87)
• Memory-Augmented Re-Completion for 3D Semantic Scene Completion (AAAI 2025 · similarity 0.86)
• H2GFormer: Horizontal-to-Global Voxel Transformer for 3D Semantic Scene Completion (AAAI 2024 · similarity 0.86)
• LT3SD: Latent Trees for 3D Scene Diffusion (CVPR 2025 · similarity 0.86)
• Scaling Diffusion Models to Real-World 3D LiDAR Scene Completion (CVPR 2024 · similarity 0.86)