Research map / Point clouds / Segmentation and detection
Voxel Proposal Network via Multi-Frame Knowledge Distillation for Semantic Scene Completion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes VPNet, a dual-branch (BEV + 3D) semantic scene completion network with Confident Voxel Proposal modules that learn voxel offsets, trained with Multi-Frame Knowledge Distillation from a multi-frame teacher to a single-frame student.
Problem
Semantic scene completion from LiDAR point clouds suffers from large-scale information loss, geometric incompletion, and voxel semantic uncertainty, especially in dynamic multi-frame sequences.
Approach
Confident Voxel Proposal learns multiple branches of random-noise-conditioned offsets to propagate occupied-voxel features and model multiple semantic possibilities per voxel; Multi-Frame Knowledge Distillation transfers the fused multi-frame CVP features into a single-frame network in two stages (super-voxel KL for branches, plus fused-feature distillation).
Key results
- On SemanticKITTI online test, VPNet with MFKD reaches 60.7% IoU and 25.6% mIoU, versus 58.8% IoU and 23.5% mIoU for SSA-SC.
- On SemanticPOSS validation, VPNet with MFKD achieves 57.3% IoU and 23.3% mIoU, outperforming SSA-SC (53.3% IoU, 21.6% mIoU).
- Multi-frame teacher with frames t/t+2/t+4 gives the best validation result (61.1% IoU, 26.8% mIoU) versus 59.5% IoU for single frame t/t/t.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| adding the 3D completion subnetwork (without CVP) to BEV+segmentation branches (ablation) | dual-branch VPNet · SemanticKITTI validation | IoU: +0.8 IoU (58.3 to 59.1) | Helped |
| adding Confident Voxel Proposal (CVP) to the dual-branch network (ablation) | dual-branch VPNet · SemanticKITTI validation | mIoU: +0.7 mIoU (24.9 to 25.6) | Helped |
| random noise channel number Cz set to 0 in CVP (ablation) | CVP · SemanticKITTI validation | mIoU: -0.6 mIoU vs Cz=4 (25.0 vs 25.6) | Hurt |
| increasing random noise channels beyond 4 (6 or 8) in CVP (ablation) | CVP · SemanticKITTI validation | mIoU: -0.3 to -0.7 mIoU vs Cz=4 | Hurt |
| number of CVP branches Q increased from 1 to 3 (ablation) | CVP · SemanticKITTI validation | mIoU: 25.1 to 25.6 mIoU (Q=1 to Q=3) | Helped |
| CVP branch number Q increased to 4 (ablation) | CVP · SemanticKITTI validation | IoU: 59.2 vs 59.3 at Q=3 | No clear effect |
| weighted multi-branch fusion instead of addition in CVP (ablation) | CVP · SemanticKITTI validation | mIoU: +0.2 mIoU (25.4 to 25.6) | Helped |
| multi-frame teacher using frames t/t+2/t+4 instead of adjacent frames t/t+1/t+2 (ablation) | MFKD · SemanticKITTI validation | mIoU: 26.8 vs 26.3 mIoU; 61.1 vs 60.2 IoU | Helped |
| multi-frame teacher with frames t/t+3/t+6 (larger intervals) (ablation) | MFKD · SemanticKITTI validation | mIoU: IoU 61.4 (up from 61.1) but mIoU 26.6 (down from 26.8) | Mixed |
| stage-1 distillation without super voxel partition (ablation) | MFKD · SemanticKITTI validation | mIoU: 25.5 mIoU vs 25.6 baseline single-frame | No clear effect |
| adding stage-1 super-voxel distillation plus stage-2 distillation (full MFKD) (ablation) | MFKD · SemanticKITTI validation | mIoU: 26.1 mIoU vs 25.6 without distillation (59.6 vs 59.3 IoU) | Helped |
| stage-2 distillation only (ablation) | MFKD · SemanticKITTI validation | mIoU: 25.8 mIoU vs 25.6 (59.6 vs 59.3 IoU) | Helped |
Surprising findings
- Using adjacent frames t/t+1/t+2 gives less gain than frames spaced further apart, which the authors attribute to insufficient supplementary information in adjacent frames.
- Plain distillation stage-1 without super-voxel partition gives almost no gain over the single-frame network (59.5% IoU, 25.5% mIoU), which the authors attribute to distraction of offset learning.
Limitations
- Fine-grained geometric shape learning is limited by a single round of offset learning and feature propagation.
- Gains over strong baselines are small (e.g., +0.3% IoU from MFKD on SemanticKITTI test).
Experimental setup
Datasets: SemanticKITTI SemanticPOSS
Benchmarks: SemanticKITTI SemanticPOSS
Metrics: IoU mIoU
Baselines: SSCNet SSCNet-full TS3D LMSCNet LMSCNet-SS Local-DIFs JS3C-Net SSA-SC KD PVKD DSKD
Builds on: Cylinder3D SSCNet PVKD DSKD SMF-SSD
Closest papers by meaning
- H2GFormer: Horizontal-to-Global Voxel Transformer for 3D Semantic Scene Completion (AAAI 2024 · similarity 0.89)
- Context and Geometry Aware Voxel Transformer for Semantic Scene Completion (NeurIPS 2024 · similarity 0.89)
- Not All Voxels Are Equal: Hardness-Aware Semantic Scene Completion with Self-Distillation (CVPR 2024 · similarity 0.88)
- VLScene: Vision-Language Guidance Distillation for Camera-Based 3D Semantic Scene Completion (AAAI 2025 · similarity 0.88)
- Unleashing Network Potentials for Semantic Scene Completion (CVPR 2024 · similarity 0.87)
- Memory-Augmented Re-Completion for 3D Semantic Scene Completion (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.