Research map / Detection and segmentation / 3D detection and driving
Learning to Detect Objects from Multi-Agent LiDAR Scans without Manual Labels
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DOtA, an unsupervised 3D object detector trained only from internally shared ego-pose/shape of multi-agent LiDAR, filtering preliminary labels with multi-scale box encoding and refining features with label-internal contrastive learning.
Problem
Unsupervised 3D object detection from LiDAR produces low-quality clustering-based pseudo-labels due to sparsity and limited views; multi-agent collaborative data could help but has not been used without manual labels.
Approach
Initialize a detector from unlabeled shared agent poses/shapes (free boxes of agents), then filter the noisy preliminary labels using multi-view collision, boundary-alignment and distance-confidence encodings, and use the high/low-quality labels as positive/negative prompts in an InfoNCE contrastive loss.
Key results
- On V2V4Real, DOtA reaches AP@0.3 54.60, AP@0.5 48.84, AP@0.7 22.41, outperforming the best prior unsupervised method (CPD with internal shared-info: 40.67/30.27/4.73) and improving average performance by 16.83%.
- DOtA added to AttFuse, V2X-ViT and Where2Comm improves average AP by 32.61% on V2V4Real and 35.35% on OPV2V relative to internal-shared-info-only baselines, per the paper.
- On OPV2V, DOtA label precision/recall at IoU 0.5 is 79.34/65.74 at IoU 0.3 vs CPD's 59.31/56.23 at IoU 0.3 (Table 5).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace CPD with DOtA | AttFuse + PointPillars · V2V4Real | AP@0.3: 40.67 -> 54.60 | Helped |
| replace CPD with DOtA | AttFuse + PointPillars · V2V4Real | AP@0.7: 4.73 -> 22.41 | Helped |
| replace CPD with DOtA | AttFuse + PointPillars · OPV2V | AP@0.3: 59.17 -> 66.14 | Helped |
| replace CPD with DOtA | AttFuse + PointPillars · OPV2V | AP@0.7: 27.72 -> 24.57 | Hurt |
| add Collision Probability Encoding (ablation) | AttFuse + self-training on OPV2V · OPV2V | AP@0.3: 30.02 -> 55.62 | Helped |
| add Boundary Alignment Encoding (ablation) | AttFuse + self-training on OPV2V · OPV2V | AP@0.3: 30.02 -> 43.28 | Helped |
| add all three MBE encodings (ablation) | AttFuse + self-training on OPV2V · OPV2V | AP@0.3: 30.02 -> 60.96 | Helped |
| add Information Confidence Encoding to CPE + BAE (ablation) | AttFuse + self-training on OPV2V · OPV2V | AP@0.3: 58.75 -> 60.96 | Helped |
| add Label-Internal Contrastive Learning to MBE (ablation) | DOtA on OPV2V · OPV2V | AP@0.3: 60.96 -> 66.14 | Helped |
| add Label-Internal Contrastive Learning to MBE (ablation) | DOtA on OPV2V · OPV2V | AP@0.5: 45.77 -> 52.37 | Helped |
| set LICL loss weight γ to 1.0 (ablation) | DOtA · OPV2V | AP@0.3: 0.61 -> 0.66 | Helped |
Surprising findings
- Using a low confidence threshold (0.01) gives high recall but strongly degrades label precision, motivating filtering.
- On OPV2V at IoU 0.7, DOtA lags the prior best method CPD, which the authors attribute to the simulated dataset having less noise favoring cluster-based methods.
Limitations
- Performance at strict IoU 0.7 on simulated OPV2V trails CPD.
- Relies on agents sharing ego-pose and ego-shape and pre-registered vehicle types.
- Hyperparameters such as collision and alignment tolerances (0.1, 0.7) and scaling factors are set empirically.
Experimental setup
Datasets: V2V4Real OPV2V V2X-Real
Benchmarks: V2V4Real OPV2V V2X-Real
Metrics: AP@0.3 AP@0.5 AP@0.7 precision recall
Baselines: DBSCAN MODEST OYSTER CPD LISO AttFuse V2X-ViT Where2Comm
Builds on: MODEST OYSTER CPD AttFuse PointPillars
Closest papers by meaning
- DI-V2X: Learning Domain-Invariant Representation for Vehicle-Infrastructure Collaborative 3D Object Detection (AAAI 2024 · similarity 0.87)
- UNION: Unsupervised 3D Object Detection using Object Appearance-based Pseudo-Classes (NeurIPS 2024 · similarity 0.86)
- MixSup: Mixed-grained Supervision for Label-efficient LiDAR-based 3D Object Detection (ICLR 2024 · similarity 0.86)
- iDet3D: Towards Efficient Interactive Object Detection for LiDAR Point Clouds (AAAI 2024 · similarity 0.86)
- A-Teacher: Asymmetric Network for 3D Semi-Supervised Object Detection (CVPR 2024 · similarity 0.86)
- CMDA: Cross-Modal and Domain Adversarial Adaptation for LiDAR-Based 3D Object Detection (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.