Research map / 3D vision / 3D generation
Not All Voxels Are Equal: Hardness-Aware Semantic Scene Completion with Self-Distillation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HASSC, a hard voxel mining head with EMA self-distillation that improves camera-based semantic scene completion baselines (VoxFormer, StereoScene) on SemanticKITTI without extra inference cost.
Problem
Existing semantic scene completion methods treat all voxels equally in voxel-wise classification, so hard voxels (boundaries, challenging regions) are under-optimized, while most voxels are easy empty ones that consume computation.
Approach
Select hard voxels dynamically using a global hardness (inverse of top-1 minus top-2 class probability margin from the coarse prediction), weight their refinement losses by a local hardness derived from local geometric anisotropy (semantic disagreement with 6 neighbors), and add an EMA teacher for self-distillation; the whole scheme is dropped at inference.
Key results
- HASSC-VoxFormer-T reaches 14.74% mIoU vs 13.33% for the VoxFormer-T baseline on the SemanticKITTI validation set, with IoU 44.58% vs 44.16%.
- On the SemanticKITTI hidden test set, HASSC-VoxFormer-T achieves 14.38% mIoU, above the listed camera-based methods including NDC-Scene (12.58%).
- Inference time is essentially unchanged (720.84 ms vs 724.05 ms) with only 0.9% more parameters during training.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add HASSC to VoxFormer-T | VoxFormer-T · SemanticKITTI val, long range (L) | mIoU: 13.35 -> 14.74 | Helped |
| Add HASSC to VoxFormer-T | VoxFormer-T · SemanticKITTI val, short range (S) | mIoU: +2.55%mIoU | Helped |
| Add HASSC to VoxFormer-T | VoxFormer-T · SemanticKITTI hidden test | mIoU: +0.97%mIoU | Helped |
| Add HASSC to VoxFormer-S | VoxFormer-S · SemanticKITTI hidden test | mIoU: +1.14%mIoU | Helped |
| Use global hardness only (ablation) | VoxFormer-T · SemanticKITTI val, long range (L) | mIoU: 13.33 -> 13.30 | No clear effect |
| Use local hardness only (ablation) | VoxFormer-T · SemanticKITTI val, long range (L) | mIoU: 13.33 -> 13.40 | No clear effect |
| Combine global and local hardness (ablation) | VoxFormer-T · SemanticKITTI val, long range (L) | mIoU: 13.33 -> 13.91 | Helped |
| Add teacher-guided hard voxel mining (T-HVM) (ablation) | HVM with global and local hardness · SemanticKITTI val, long range (L) | mIoU: 13.91 -> 14.03 | Helped |
| Add teacher-branch self-distillation (ablation) | HVM-based VoxFormer-T · SemanticKITTI val, long range (L) | mIoU: 14.03 -> 14.74 | Helped |
| Set hard voxel count N to 1024 (ablation) | HVM-based VoxFormer-T · SemanticKITTI val, long range (L) | mIoU: 13.33 -> 13.52 | Helped |
| Set hard voxel count N to 2048 (ablation) | HVM-based VoxFormer-T · SemanticKITTI val, long range (L) | mIoU: 13.33 -> 13.64 | Helped |
| Set hard voxel count N to 8192 instead of 4096 (ablation) | HVM-based VoxFormer-T · SemanticKITTI val, long range (L) | mIoU: 14.03 -> 13.74 | Hurt |
Surprising findings
- HASSC-VoxFormer-S with a single image outperforms VoxFormer-T using 5 frames on validation mIoU (13.48% vs 13.35%).
- Prior 2D/indoor hard-sample methods (PALNet, PointRend, Xiao et al.) yield only marginal gains when transplanted to large-scale 3D outdoor scenes.
Limitations
- Gains are modest in absolute terms (about +1.1 to +1.4 mIoU on validation for VoxFormer baselines).
- Evaluated only on SemanticKITTI with camera-based baselines; no evaluation on other SSC datasets.
- Hyperparameters (N, lambda, alpha, beta, delta) tuned via ablations on the validation set.
Experimental setup
Datasets: SemanticKITTI
Benchmarks: SemanticKITTI
Metrics: IoU mIoU
Baselines: VoxFormer-S VoxFormer-T StereoScene MonoScene TPVFormer OccFormer NDC-Scene PALNet PointRend Xiao et al. hardness learning
Builds on: VoxFormer PointRend PALNet (local geometric anisotropy) mean teacher
Closest papers by meaning
- Three Cars Approaching within 100m! Enhancing Distant Geometry by Tri-Axis Voxel Scanning for Camera-based Semantic Scene Completion (CVPR 2025 · similarity 0.89)
- OPUS: Occupancy Prediction Using a Sparse Set (NeurIPS 2024 · similarity 0.89)
- Memory-Augmented Re-Completion for 3D Semantic Scene Completion (AAAI 2025 · similarity 0.89)
- Achieving Speed-Accuracy Balance in Vision-based 3D Occupancy Prediction via Geometric-Semantic Disentanglement (AAAI 2025 · similarity 0.89)
- CSV-Occ: Fusing Multi-frame Alignment for Occupancy Prediction with Temporal Cross State Space Model and Central Voting Mechanism (ICML 2025 · similarity 0.89)
- OctOcc: High-Resolution 3D Occupancy Prediction with Octree (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.