Research map / Detection and segmentation / 3D detection and driving
SeMoLi: What Moves Together Belongs Together
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SeMoLi, a data-driven message-passing-network approach that learns to cluster long-term motion trajectories into moving-object instances, then pseudo-labels Lidar data to train a PointPillars detector without manual boxes.
Problem
Semi-supervised 3D Lidar object detection using motion cues: heuristic density-based clustering used for pseudo-labeling of moving objects is hand-tuned, noisy, and cannot improve with more data.
Approach
Revisit correlation clustering within message passing networks so that a learned edge classifier groups point trajectories (spatial and motion features) into class-agnostic object instances; per-cluster boxes are inflated and used as pseudo-labels for a student detector.
Key results
- SeMoLi reaches 57.5 AP on moving objects without labels versus 43.2 for DBSCAN++-based pseudo-labels (+14 improvement claimed in abstract).
- SeMoLi pseudo-label F1 at 0.4 3DIoU rises from 50.4 (scene flow variant) to 57.6 with 90% training data, versus 16.2 for DBSCAN++ with outlier filtering.
- Cross-dataset: SeMoLi trained on Waymo and applied to Argoverse2 gives 57.6 AP on moving objects versus 82.9 for the supervised detector.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SeMoLi rather than DBSCAN++† pseudo-labels to train PointPillars without labeled data | PointPillars · Waymo Open val, moving objects | AP@0.4: 43.2 -> 57.5 | Helped |
| Train SeMoLi with more training data (10% to 90%) | SeMoLi · Waymo val pseudo, 3DIoU@0.4 | F1: 53.1 -> 57.6 | Helped |
| Add position features to velocity-only node features (ablation) | SeMoLi 10 · Waymo val pseudo, SegIoU@0.4 | F1: 67.1 -> 70.9 | Helped |
| Use position-only rather than velocity and position node features (ablation) | SeMoLi 10 · Waymo val pseudo, SegIoU@0.4 | F1: 70.9 -> 67.5 | Hurt |
| Use velocity-only rather than position edge features (ablation) | SeMoLi 10 · Waymo val pseudo, SegIoU@0.4 | F1: 70.9 -> 62.9 | Hurt |
| Add velocity to position-based edge features (ablation) | SeMoLi 10 · Waymo val pseudo, SegIoU@0.4 | F1: 70.9 -> 70.9 | No clear effect |
| Inflate tight cluster bounding boxes to minimum dimensions (ablation) | SeMoLi 10 · Waymo val pseudo, 3DIoU@0.4 | F1: 30.3 -> 53.1 | Helped |
| Use position-based rather than velocity-based k-nearest-neighbor graph construction (ablation) | SeMoLi · Waymo val pseudo, oracle SegIoU@0.4 | F1: 51.4 -> 90.9 | Helped |
| Use long-term trajectory cues rather than local scene-flow cues (ablation) | SeMoLi · Waymo val pseudo, 3DIoU@0.4 | F1: 50.4 -> 53.1 | Helped |
| Train PointPillars using SeMoLi rather than DBSCAN++† pseudo-labels | PointPillars · Waymo Open val, all objects | AP@0.4: 14.9 -> 19.5 | Helped |
| Apply the Waymo-trained SeMoLi pseudo-labeler to Argoverse2 | PointPillars · Argoverse2, moving objects | AP@0.4: 57.6 | Helped |
Surprising findings
- Pseudo-labels from SeMoLi with 10% labeled data outperform the detector trained on ground-truth moving-only labels in the all-objects setting, possibly because noisy flow retains some static objects.
- Oracle DBSCAN with ground-truth motion gives strong pseudo-labels (69.0 F1@0.4), but with estimated motion DBSCAN variants collapse to around 16 F1, while SeMoLi keeps above 50.
Limitations
- Evaluation is class-agnostic only; fine-grained semantics are not produced by pseudo-labels.
- Relies on estimated trajectories from a self-supervised flow network, causing noisy precision for pseudo-labels.
- Pseudo-label recall and precision on Argoverse2 drop substantially compared with Waymo results.
Experimental setup
Datasets: Waymo Open Dataset Argoverse2
Benchmarks: Waymo Open Dataset validation Argoverse2
Metrics: F1 score precision recall AP mAP
Baselines: DBSCAN DBSCAN++ DBSCAN++ with long-term velocity (DBSCAN++l) SeMoLi sf (scene flow variant) PointPillars fully supervised
Builds on: DBSCAN++ Message Passing Network correlation clustering PointPillars DBSCAN
Closest papers by meaning
- Seg2Box: 3D Object Detection by Point-Wise Semantics Supervision (AAAI 2025 · similarity 0.88)
- MixSup: Mixed-grained Supervision for Label-efficient LiDAR-based 3D Object Detection (ICLR 2024 · similarity 0.88)
- UNION: Unsupervised 3D Object Detection using Object Appearance-based Pseudo-Classes (NeurIPS 2024 · similarity 0.87)
- 3DSFLabelling: Boosting 3D Scene Flow Estimation by Pseudo Auto-labelling (CVPR 2024 · similarity 0.87)
- Semi-supervised 3D Object Detection with PatchTeacher and PillarMix (AAAI 2024 · similarity 0.87)
- Motion Cues from Image-based Point Tracking for LiDAR Scene Flow Estimation (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.