Research map / Label-efficient and robust learning / Self-supervised and contrastive
Self-Supervised Bird’s Eye View Motion Prediction with Cross-Modality Signals
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a cross-modality self-supervised framework for BEV motion prediction that uses camera optical flow to supervise LiDAR-based motion via a masked Chamfer loss, piecewise rigidity loss, and temporal consistency loss.
Problem
Self-supervised BEV motion flow prediction from LiDAR point clouds relies on point correspondences, causing fake flow on static background and inconsistent flow within objects due to sparsity and noise.
Approach
Uses sequential camera optical flow to derive a pseudo static/dynamic mask (masking the Chamfer loss to dynamic points), over-segmented optical-flow image pieces projected to points as rigid-piece constraints, and a temporal consistency loss across predicted frames; the image data is used only during training.
Key results
- On nuScenes test, the method achieves the best self-supervised results, improving fast-speed (>5 m/s) mean error to 2.0766 from 3.5266 for ContrastMotion, a roughly 41% improvement.
- Masked Chamfer loss improves static-motion mean error from 0.2515 to 0.0704 in the ablation (Exp. 1 vs 3).
- On Argoverse2, the full method reaches static mean error 0.0736 and fast-speed 2.3359, reported as comparable to supervised learning.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use cross-modality self-supervision with masked Chamfer, piecewise rigidity, and temporal consistency | MotionNet · nuScenes test, Speed > 5 m/s | Mean error: 41% | Helped |
| Use cross-modality self-supervision with masked Chamfer, piecewise rigidity, and temporal consistency | MotionNet · nuScenes test, Speed ≤ 5 m/s | Mean error: 7% | Helped |
| Use cross-modality self-supervision with masked Chamfer, piecewise rigidity, and temporal consistency | MotionNet · nuScenes test, Static | Mean error: 38% | Helped |
| Add masked Chamfer distance loss instead of original Chamfer distance (ablation) | Self-supervised MotionNet · nuScenes ablation, Static | Error: 0.2515 -> 0.0704; 0.2097 -> 0.0514 | Helped |
| Add piecewise rigidity loss (ablation) | Self-supervised MotionNet · nuScenes ablation, Static | Error: 0.0704 -> 0.0514 | Helped |
| Replace local smoothness regularization with piecewise rigidity loss (ablation) | Self-supervised MotionNet · nuScenes ablation, Speed > 5 m/s | Error: 2.2142 -> 2.0766 | Helped |
| Add temporal consistency loss (ablation) | Self-supervised MotionNet · nuScenes temporal ablation, Static | Error: 0.1150 -> 0.0838; 0.5549 -> 0.4212 | Helped |
| Add backward-frame prediction (frame -1) (ablation) | Self-supervised MotionNet · nuScenes temporal ablation, Speed > 5 m/s | Error: 2.8830 -> 2.0766 | Helped |
| Add further-future prediction (frame 2) (ablation) | Self-supervised MotionNet · nuScenes temporal ablation, Speed > 5 m/s | Error: 2.1814 -> 2.0766 | Mixed |
Surprising findings
- Performance degrades on nighttime and adverse-weather scenes because pseudo labels depend on camera images, and the authors did not exclude such scenes during training.
Limitations
- Pseudo labels rely on camera images, so quality drops at night and in adverse weather.
- Requires synchronized multi-view camera video during training and optical flow estimation, though not at inference.
Experimental setup
Datasets: nuScenes Argoverse2
Benchmarks: nuScenes Argoverse2
Metrics: mean error median error
Baselines: PillarMotion ContrastMotion WeakMotionNet HPLFlowNet PointRCNN LSTM-ED MotionNet BE-STI FlowNet3D
Builds on: MotionNet Chamfer distance loss ContrastMotion PillarMotion
Closest papers by meaning
- Self-Supervised Class-Agnostic Motion Prediction with Spatial and Temporal Consistency Regularizations (CVPR 2024 · similarity 0.87)
- Moving Off-the-Grid: Scene-Grounded Video Representations (NeurIPS 2024 · similarity 0.84)
- Enhancing End-to-End Autonomous Driving with Latent World Model (ICLR 2025 · similarity 0.84)
- Self-Supervised Learning of Motion Concepts by Optimizing Counterfactuals (NeurIPS 2025 · similarity 0.84)
- BEV-MAE: Bird’s Eye View Masked Autoencoders for Point Cloud Pre-training in Autonomous Driving Scenarios (AAAI 2024 · similarity 0.84)
- H-Flow: Self-supervised Human Scene Flow via Physics-inspired Joint Multi-modal Learning (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.