Research map / Point clouds / Segmentation and detection
BEV-MAE: Bird’s Eye View Masked Autoencoders for Point Cloud Pre-training in Autonomous Driving Scenarios
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BEV-MAE, a masked autoencoder pre-training method for LiDAR 3D encoders in autonomous driving that masks BEV grids, uses a shared learnable point token, and adds point density prediction.
Problem
Self-supervised pre-training of LiDAR 3D object detectors for autonomous driving, where prior masked point modeling methods are designed for indoor/synthetic data or use voxel-based masking that mismatches BEV-based detectors.
Approach
Mask non-empty BEV grids so the 3D encoder learns BEV-aligned features with a one-layer 3x3 conv decoder (no upsampling), replace masked points with a shared learnable point token to keep receptive field consistent with fine-tuning, and predict point density of masked grids to encode distance/location cues.
Key results
- On nuScenes test, BEV-MAE with TransFusion-L achieves 73.6 NDS and 69.6 mAP, new state of the art among LiDAR detectors reported.
- On Waymo validation with 20% fine-tuning data, BEV-MAE reaches 66.70 L2 mAP / 64.25 L2 APH, outperforming ProposalContrast (65.76/62.49-level baselines) and GD-MAE at lower pre-training cost.
- With 5% labeled Waymo data, BEV-MAE surpasses the from-scratch baseline by 7.22 mAP and 7.43 APH.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Pre-train with BEV-MAE rather than GD-MAE | VoxelNet + CenterPoint · Waymo validation, 100% pre-training data | L2 mAP: 66.98 -> 67.02 | Helped |
| Pre-train with BEV-MAE rather than GD-MAE | VoxelNet + CenterPoint · Waymo validation, 100% pre-training data | L2 APH: 64.53 -> 64.55 | Helped |
| Pre-train with BEV-MAE rather than GeoMAE | TransFusion-L · nuScenes test | NDS: 72.5 -> 73.6 | Helped |
| Pre-train with BEV-MAE rather than GeoMAE | TransFusion-L · nuScenes test | mAP: 67.8 -> 69.6 | Helped |
| Use BEV-MAE pre-training rather than random initialization | CenterPoint · Waymo, 5% labeled data | L2 mAP: 44.41 -> 51.63 | Helped |
| Add shared learnable point token (ablation) | BEV-MAE with normalized coordinates and density prediction · Waymo, 20% labeled data | L2 mAP: 66.49 -> 66.70 | Helped |
| Normalize coordinates in point reconstruction (ablation) | BEV-MAE with coordinate reconstruction and learnable token · Waymo, 20% labeled data | L2 mAP: 65.66 -> 66.20 | Helped |
| Add density prediction to normalized coordinate reconstruction (ablation) | BEV-MAE with learnable token · Waymo, 20% labeled data | L2 mAP: 66.20 -> 66.70 | Helped |
| Replace density prediction with number-of-points prediction (ablation) | BEV-MAE with normalized coordinates and learnable token · Waymo, 20% labeled data | L2 mAP: 66.70 -> 65.32 | Hurt |
| Replace one-layer 3×3 Conv decoder with Residual Conv block (ablation) | BEV-MAE · Waymo, 20% labeled data | L2 mAP: 66.70 -> 66.61 | Hurt |
| Replace one-layer 3×3 Conv decoder with Transformer block (ablation) | BEV-MAE · Waymo, 20% labeled data | L2 mAP: 66.70 -> 65.80 | Hurt |
| Replace BEV-guided masking with voxel masking (ablation) | BEV-MAE · Waymo, 20% labeled data | L2 mAP: 66.70 -> 66.63 | Hurt |
Surprising findings
- Pre-training gains are marginal when fine-tuning on the full Waymo dataset, attributed to detector capacity being the bottleneck.
- A simple one-layer 3x3 convolution decoder outperforms more complex residual conv and transformer decoders while costing less.
Limitations
- Marginal gains when fine-tuning on the full Waymo dataset.
- Nuscenes SOTA result uses a modified model structure with larger encoder channels and layers, not the standard configuration.
Experimental setup
Datasets: nuScenes Waymo Open Dataset
Benchmarks: nuScenes Waymo Open Dataset
Metrics: mAP NDS L2 mAP L2 APH
Baselines: PointContrast DepthContrast Point-M2AE ProposalContrast MSP GD-MAE GCC-3D MV-JAR GeoMAE Link TransFusion-L CenterPoint
Builds on: Masked autoencoders Point-MAE GD-MAE Voxel-MAE TransFusion-L
Closest papers by meaning
- M-BEV: Masked BEV Perception for Robust Autonomous Driving (AAAI 2024 · similarity 0.90)
- BEV@DC: Bird's-Eye View Assisted Training for Depth Completion (CVPR 2023 · similarity 0.90)
- BEV-SAN: Accurate BEV 3D Object Detection via Slice Attention Networks (CVPR 2023 · similarity 0.89)
- CALICO: Self-Supervised Camera-LiDAR Contrastive Pre-training for BEV Perception (ICLR 2024 · similarity 0.88)
- Multi-Scale Neighborhood Occupancy Masked Autoencoder for Self-Supervised Learning in LiDAR Point Clouds (CVPR 2025 · similarity 0.88)
- 3D Feature Prediction for Masked-AutoEncoder-Based Point Cloud Pretraining (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.