Research map / Point clouds / Point cloud classification
Self-Supervised Class-Agnostic Motion Prediction with Spatial and Temporal Consistency Regularizations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Self-supervised class-agnostic motion prediction from unlabeled LiDAR, using optimal-transport pseudo motion labels plus cluster, forward, and backward consistency regularizations on MotionNet.
Problem
Class-agnostic motion prediction from LiDAR typically needs costly annotations; prior annotation-efficient methods rely on weak labels or image data, leaving self-supervised motion prediction from point cloud sequences underexplored.
Approach
Generate coarse pseudo motion labels via optimal transport between current and future BEV cells (with pre-warping by the model's own predictions), then regularize training with a cluster-level spatial consistency loss and forward/backward temporal consistency losses.
Key results
- On nuScenes, the method reduces mean error versus self-supervised PillarMotion by 74.1% at static, 53.9% at slow, and 35.4% at fast speeds (fast mean error 2.2943 m).
- Ablation shows adding cluster, forward, and backward regularizations to the pseudo-label baseline lowers fast-speed mean error from 3.4503 to 2.2943 m.
- On Waymo, the method reports mean errors of about 0.05 m static, 0.44 m slow, and 2.37 m fast, outperforming self-supervised WeakMotionNet.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use spatial and temporal consistency regularizations with pseudo-label training (ablation) | Self-supervised MotionNet · nuScenes static, slow, and fast | mean error (m): 0.1650 -> 0.0419; 0.5540 -> 0.3213; 3.4503 -> 2.2943 | Helped |
| Add cluster consistency regularization (ablation) | Self-supervised MotionNet · nuScenes static, slow, and fast | mean error (m): 0.1650 -> 0.1705; 0.5540 -> 0.4154; 3.4503 -> 2.7027 | Mixed |
| Add cluster consistency regularization to the model with forward and backward losses (ablation) | Self-supervised MotionNet · nuScenes static, slow, and fast | mean error (m): 0.0502 -> 0.0419; 0.3432 -> 0.3213; 2.5154 -> 2.2943 | Helped |
| Add forward consistency regularization (ablation) | Self-supervised MotionNet · nuScenes static, slow, and fast | mean error (m): 0.0499 -> 0.0419; 0.3277 -> 0.3213; 2.5753 -> 2.2943 | Helped |
| Add backward consistency regularization (ablation) | Self-supervised MotionNet · nuScenes static, slow, and fast | mean error (m): 0.1450 -> 0.0419; 0.4087 -> 0.3213; 2.6509 -> 2.2943 | Helped |
| Replace KNN consistency loss with cluster consistency loss (ablation) | Self-supervised MotionNet · nuScenes | mean error (m): K=3: 0.0371, 0.3370, 2.5362; K=5: 0.0490, 0.3378, 2.4568; K=10: 0.0731, 0.3405, 2.5116; dc=3: 0.0419, 0.3213, 2.2943 | Mixed |
| Vary cluster distance dc (ablation) | Self-supervised MotionNet · nuScenes | mean error (m): dc=2: 0.0526, 0.3438, 2.1814; dc=3: 0.0419, 0.3213, 2.2943; dc=4: 0.0459, 0.3318, 2.4434 | Mixed |
| Vary backward-loss temperature θb (ablation) | Self-supervised MotionNet · nuScenes | mean error (m): θb=5: 0.0482, 0.3264, 2.3580; θb=10: 0.0419, 0.3213, 2.2943; θb=15: 0.0293, 0.3091, 2.4149; −: 0.0382, 0.3298, 2.4596 | Mixed |
| Use the proposed self-supervised method instead of PillarMotion | Class-agnostic motion prediction · nuScenes static, slow, and fast | mean error (m): 0.1620 -> 0.0419; 0.6972 -> 0.3213; 3.5504 -> 2.2943 | Helped |
| Use the proposed self-supervised method instead of WeakMotion self-supervised | Class-agnostic motion prediction · Waymo static, slow, and fast | mean error (m): 0.1345 -> 0.0515; 0.5833 -> 0.4440; 3.0180 -> 2.3692 | Helped |
Surprising findings
- Self-supervised method without any annotations surpasses the weakly supervised WeakMotionNet at static and slow speed levels on nuScenes.
Limitations
- Remaining ground points, noise, and cells without correspondences still degrade pseudo labels.
- Breadth-first clustering has under-clustering (9.6%) and over-clustering (11.1%) rates on nuScenes.
- Self-supervised approach still has a performance gap to fully-supervised MotionNet at fast speeds.
Experimental setup
Datasets: nuScenes Waymo
Benchmarks: nuScenes Waymo
Metrics: mean error median error
Baselines: FlowNet3D HPLFlowNet LSTM-ED MotionNet BE-STI WeakMotionNet PointPWC RigidFlow PillarMotion
Builds on: MotionNet Sinkhorn algorithm WeakMotionNet PillarMotion
Closest papers by meaning
- Semi-supervised Class-Agnostic Motion Prediction with Pseudo Label Regeneration and BEVMix (AAAI 2024 · similarity 0.89)
- UnO: Unsupervised Occupancy Fields for Perception and Forecasting (CVPR 2024 · similarity 0.88)
- Self-Supervised Bird’s Eye View Motion Prediction with Cross-Modality Signals (AAAI 2024 · similarity 0.87)
- SmartPretrain: Model-Agnostic and Dataset-Agnostic Representation Learning for Motion Prediction (ICLR 2025 · similarity 0.86)
- 3DSFLabelling: Boosting 3D Scene Flow Estimation by Pseudo Auto-labelling (CVPR 2024 · similarity 0.86)
- Motion Cues from Image-based Point Tracking for LiDAR Scene Flow Estimation (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.