Research map / 3D vision / 3D generation
HUNTER: Unsupervised Human-centric 3D Detection via Transferring Knowledge from Synthetic Instances to Real Scenes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HUNTER, an unsupervised LiDAR 3D human detector that inserts SMPL synthetic humans into real scenes, trains on them, and refines via pseudo-label filtering, BEV feature alignment, and skeleton supervision.
Problem
Unsupervised 3D detection of humans in LiDAR point clouds of diverse human-centric scenes, where clustering or motion-based pseudo-labels are poor due to sparse, close-together humans and static people.
Approach
Transfer knowledge from synthetic SMPL human instances to real scenes: insert them on ground, simulate LiDAR occlusion via range-image merging, mask the receptive field to avoid penalizing unlabeled real humans, then refine with tracking-filtered pseudo-labels, BEV feature alignment, and body-joint supervision.
Key results
- On HuCenLife, HUNTER reaches 62.15 mAP without labels versus 69.02 mAP for fully supervised CenterPoint and 33.09 for MODEST.
- On STCrowd, HUNTER reaches 67.20 mAP versus 21.90 for OYSTER and 17.89 for MODEST, close to the supervised 85.17.
- Fine-tuning the pretrained feature extractor with 100% ground truth gives 72.31 mAP, exceeding the fully supervised 69.02 on HuCenLife.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Transfer from synthetic human instances instead of using DBSCAN pseudo-labels (ablation) | HUNTER on HuCenLife · HuCenLife | mAP: 21.31 -> 28.36 | Helped |
| Add receptive field control (ablation) | HUNTER on HuCenLife · HuCenLife | mAP: 28.36 -> 40.03 | Helped |
| Add pseudo-label retraining without BEV feature alignment (ablation) | HUNTER on HuCenLife · HuCenLife | mAP: 40.03 -> 35.78 | Hurt |
| Add BEV feature alignment (ablation) | HUNTER on HuCenLife · HuCenLife | mAP: 35.78 -> 40.94 | Helped |
| Add receptive field update (ablation) | HUNTER on HuCenLife · HuCenLife | mAP: 55.13 -> 57.71 | Helped |
| Add human structural knowledge transfer (ablation) | HUNTER on HuCenLife · HuCenLife | mAP: 55.13 -> 59.02 | Helped |
| Add final fine-tuning on original training data (ablation) | HUNTER on HuCenLife · HuCenLife | mAP: 59.02 -> 62.15 | Helped |
| Increase synthetic training frames from 5k to 10k (ablation) | HUNTER · HuCenLife | mAP: 53.70 -> 60.16 | Helped |
| Increase synthetic training frames from 10k to 25k (ablation) | HUNTER · HuCenLife | mAP: 60.16 -> 59.53 | Hurt |
| Increase synthetic training frames from 25k to 50k (ablation) | HUNTER · HuCenLife | mAP: 59.53 -> 62.15 | Helped |
| Use HUNTER instead of OYSTER | 3D human detection · HuCenLife | mAP: 33.33 -> 62.15 | Helped |
| Use HUNTER instead of OYSTER | 3D human detection · STCrowd | mAP: 21.90 -> 67.20 | Helped |
Surprising findings
- Using only 5k synthetic frames already gives 53.70 mAP on HuCenLife, above prior unsupervised SOTA.
- Freezing the feature extractor and fine-tuning with 10% ground truth outperforms full supervised training on 100% ground truth.
Limitations
- Pseudo-labels from synthetic-only training remain lower quality, so training on them directly is ineffective without alignment and refinement.
- Relies on ground-plane placement of inserted humans and on a ground-removal/RANSAC step.
- Evaluated on only two human-centric LiDAR datasets.
Experimental setup
Datasets: HuCenLife STCrowd SURREAL SMPL
Benchmarks: HuCenLife STCrowd
Metrics: AP mAP mPrecision mRecall
Baselines: DBSCAN MODEST OYSTER CenterPoint (fully supervised)
Builds on: CenterPoint OYSTER MODEST SMPL
Closest papers by meaning
- Cross-view and Cross-pose Completion for 3D Human Understanding (CVPR 2024 · similarity 0.87)
- A Unified Framework for Human-centric Point Cloud Video Understanding (CVPR 2024 · similarity 0.85)
- ODHSR: Online Dense 3D Reconstruction of Humans and Scenes from Monocular Videos (CVPR 2025 · similarity 0.85)
- Reconstructing People, Places, and Cameras (CVPR 2025 · similarity 0.85)
- FreeMan: Towards Benchmarking 3D Human Pose Estimation under Real-World Conditions (CVPR 2024 · similarity 0.85)
- 3D Human Pose Perception from Egocentric Stereo Videos (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.