Research map / Detection and segmentation / 3D detection and driving
HilDA: Hierarchical Distillation with Diffusion for Advancing Self-Supervised LiDAR Pre-training
Maciej Wozniak, Jesper Ericsson, Hariprasath Govindarajan, Truls Nyberg, Thomas Gustafsson, Patric Jensfelt, Olov Andersson
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add temporal occupancy diffusion (ablation) | ScaLR baseline · nuScenes linear probing | mIoU: 46.36 -> 50.43 | Improved |
| add temporal occupancy diffusion (ablation) | distillation-only model · nuScenes linear probing | mIoU: 53.77 -> 55.53 | Improved |
| add multi-layer distillation (ablation) | ScaLR baseline · nuScenes linear probing | mIoU: 46.36 -> 53.77 | Improved |
| add global CLS distillation (ablation) | multi-layer distillation · nuScenes linear probing | mIoU: 53.77 -> 55.13 | Improved |
| add global CLS distillation (ablation) | distillation + diffusion · nuScenes linear probing | mIoU: 55.53 -> 56.29 | Improved |
| distill last 2 layers separately instead of final layer only (ablation) | multi-layer distillation · nuScenes linear probing | mIoU: 46.4 -> 53.8 | Improved |
| extend separate distillation to last 4 layers (ablation) | multi-layer distillation · nuScenes linear probing | mIoU: 53.8 -> 51.6 | Hurt |
| use separate matching instead of KR-style aggregation (ablation) | multi-layer distillation · nuScenes linear probing | mIoU: 51.4 -> 53.8 | Improved |
| use cosine distance instead of L2 (ablation) | distillation loss · nuScenes linear probing | mIoU: 53.8 -> 56.3 | Improved |
| use cosine distance instead of KL (ablation) | distillation loss · nuScenes linear probing | mIoU: 52.1 -> 56.3 | Improved |
| use max pooling instead of learnable pooling for CLS (ablation) | global distillation · nuScenes linear probing | mIoU: 55.9 -> 56.3 | Improved |
| use diffusion instead of ALSO decoder (ablation) | occupancy objective · nuScenes linear probing | mIoU: 54.7 -> 56.3 | Improved |
Closest papers by meaning
- VLM2Scene: Self-Supervised Image-Text-LiDAR Learning with Foundation Models for Autonomous Driving Scene Understanding (AAAI 2024 · similarity 0.90)
- Three Pillars Improving Vision Foundation Model Distillation for Lidar (CVPR 2024 · similarity 0.89)
- HiLoTs: High-Low Temporal Sensitive Representation Learning for Semi-Supervised LiDAR Segmentation in Autonomous Driving (CVPR 2025 · similarity 0.89)
- LiMoE: Mixture of LiDAR Representation Learners from Automotive Scenes (CVPR 2025 · similarity 0.87)
- UnO: Unsupervised Occupancy Fields for Perception and Forecasting (CVPR 2024 · similarity 0.87)
- OLiDM: Object-aware LiDAR Diffusion Models for Autonomous Driving (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
