Research map / 3D vision / 3D generation
H2GFormer: Horizontal-to-Global Voxel Transformer for 3D Semantic Scene Completion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes H2GFormer, a camera-based transformer for 3D semantic scene completion that diffuses visible-voxel features horizontally then globally (W2G) and adds an Internal-External position-aware loss, evaluated on SemanticKITTI.
Problem
Vision-based 3D semantic scene completion in outdoor scenes, where transformer methods treat all voxels uniformly and ignore horizontal scene variation and the differing importance of object-boundary versus interior voxels.
Approach
A horizontal window-to-global attention module first aggregates semantics along the horizontal plane via windowed self-attention per height level and then propagates them globally with deformable self-attention, combined with a hierarchical multi-scale feature allocation via a pixel decoder and an IoE-based position-aware loss that emphasizes object transition voxels.
Key results
- H2GFormer-T reaches 14.60 mIoU on the SemanticKITTI hidden test set, versus 13.41 for VoxFormer-T (the previous best, per the paper).
- On the SemanticKITTI validation set, H2GFormer-T achieves 14.29 mIoU and 44.69 IoU versus 13.35 mIoU and 44.15 IoU for VoxFormer-T.
- The paper reports a 19.7% improvement over VoxFormer-S and 8.9% over VoxFormer-T on SemanticKITTI.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use H2GFormer-T instead of VoxFormer-T | Camera-based semantic scene completion · SemanticKITTI hidden test | mIoU: 13.41 -> 14.60 | Helped |
| Use H2GFormer-S instead of VoxFormer-S | Camera-based semantic scene completion · SemanticKITTI hidden test | mIoU: 12.20 -> 13.72 | Helped |
| Add the hierarchical pixel decoder (ablation) | H2GFormer-S · SemanticKITTI val | mIoU: 13.42 -> 13.73 | Helped |
| Add the W2G module (ablation) | H2GFormer-S · SemanticKITTI val | mIoU: 12.61 -> 13.73 | Helped |
| Add IoE-PALoss (ablation) | H2GFormer-S · SemanticKITTI val | mIoU: 13.38 -> 13.73 | Helped |
| Add the hierarchical pixel decoder (ablation) | H2GFormer-T · SemanticKITTI val | mIoU: 14.19 -> 14.29 | Helped |
| Add the W2G module (ablation) | H2GFormer-T · SemanticKITTI val | mIoU: 13.69 -> 14.29 | Helped |
| Add IoE-PALoss (ablation) | H2GFormer-T · SemanticKITTI val | mIoU: 13.95 -> 14.29 | Helped |
| Use hierarchical multi-scale pixel decoding instead of original FPN allocation (ablation) | H2GFormer pixel decoder · SemanticKITTI val | mIoU: 12.23 -> 12.36 | Helped |
| Use alternating regular and shifted-window attention (ablation) | H2GFormer W2G module · SemanticKITTI val | mIoU: 12.61 -> 13.73 | Helped |
| Increase W2G module count from 2 to 4 (ablation) | H2GFormer W2G module · SemanticKITTI val | mIoU: 13.73 -> 13.58 | Hurt |
| Use only 1/4-resolution features rather than the hierarchical pixel decoder (ablation) | H2GFormer pixel decoder · SemanticKITTI val | mIoU: 10.67 -> 12.36 | Hurt |
Surprising findings
- Removing the pixel decoder slightly improves IoU for H2GFormer-T (44.90 vs 44.69), although mIoU drops.
Limitations
- Evaluated only on SemanticKITTI.
- Gains over VoxFormer on IoU are small; mIoU gains are the main improvement.
- Hardware and training compute are not reported.
Experimental setup
Datasets: SemanticKITTI
Benchmarks: SemanticKITTI
Metrics: IoU mIoU
Baselines: VoxFormer-S VoxFormer-T MonoScene SurroundOcc Symphonies TPVFormer OccFormer LMSCNet
Builds on: VoxFormer Mask2Former Deformable DETR PALoss Swin Transformer
Closest papers by meaning
- Context and Geometry Aware Voxel Transformer for Semantic Scene Completion (NeurIPS 2024 · similarity 0.93)
- Semi-supervised 3D Semantic Scene Completion with 2D Vision Foundation Model Guidance (AAAI 2025 · similarity 0.90)
- IPFormer: Visual 3D Panoptic Scene Completion with Context-Adaptive Instance Proposals (NeurIPS 2025 · similarity 0.90)
- Voxel Proposal Network via Multi-Frame Knowledge Distillation for Semantic Scene Completion (NeurIPS 2024 · similarity 0.89)
- Three Cars Approaching within 100m! Enhancing Distant Geometry by Tri-Axis Voxel Scanning for Camera-based Semantic Scene Completion (CVPR 2025 · similarity 0.89)
- Memory-Augmented Re-Completion for 3D Semantic Scene Completion (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.