Research map / 3D vision / 3D generation
M-BEV: Masked BEV Perception for Robust Autonomous Driving
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes M-BEV, a masked view reconstruction module that randomly masks camera-view features during training and reconstructs them from neighboring views so BEV 3D detectors stay robust when cameras fail.
Problem
BEV-based camera-only 3D object detectors degrade severely when one or more surround-view cameras fail or are missing during driving.
Approach
Adapts MAE-style masked modeling to whole camera views rather than patches: randomly masks view features (with varying counts across epochs) and reconstructs them from spatio-temporally overlapping neighboring views via a Local MVR module that uses left/right adjacent-view overlap regions, with a lightweight transformer decoder kept at test time.</ask>
Key results
- With the back camera missing, M-BEV raises PETRv2 mAP from 0.2179 to 0.3206 (+10.3 points) and NDS from 0.3616 to 0.4516.
- On BEVStereo, M-BEV improves NDS and mAP across all six single-camera-missing settings (e.g., Front NDS 0.3901 to 0.4027).
- Local MVR outperforms Global MVR in all single-view missing settings, and the reported ablation shows a 76% masking ratio performs best.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add M-BEV with Local MVR | PETRv2 · nuScenes, Back camera missing | mAP: 0.2179 -> 0.3206 | Helped |
| Add M-BEV with Local MVR | PETRv2 · nuScenes, each single camera missing | mAP: Front: 0.3022 -> 0.3263; Front_Right: 0.3294 -> 0.3666; Front_Left: 0.3355 -> 0.3628; Back_Left: 0.3513 -> 0.3694; Back_ | Helped |
| Add M-BEV with Local MVR | BEVStereo · nuScenes, each single camera missing | mAP: Front: 0.2462 -> 0.2667; Front_Right: 0.2832 -> 0.3004; Front_Left: 0.2795 -> 0.2967; Back: 0.2373 -> 0.2580; Back_Left: | Helped |
| Train with M-BEV and Local MVR | PETRv2 · nuScenes, no camera missing | mAP: 0.3977 -> 0.4039 | Helped |
| Use Local MVR instead of Global MVR (ablation) | M-BEV (PETRv2) · nuScenes, Front camera missing | mAP: 0.3160 -> 0.3263 | Helped |
| Use Local MVR instead of Global MVR (ablation) | M-BEV (PETRv2) · nuScenes, camera missing settings | mAP: Front_Right: 0.3592 -> 0.3666; Front_Left: 0.3570 -> 0.3628; Back: 0.3146 -> 0.3206; Back_Left: 0.3633 -> 0.3694; Back_R | Helped |
| Add MVR, retaining fine-tuning and RVM (ablation) | M-BEV (PETRv2) · nuScenes, one camera missing (average) | mAP: 0.3371 -> 0.3499 | Helped |
| Add fine-tuning (ablation) | M-BEV (PETRv2) · nuScenes, one camera missing (average) | mAP: 0.3130 -> 0.3499 | Helped |
| Add positional embedding (ablation) | M-BEV (PETRv2) · nuScenes, one camera missing (average) | mAP: 0.3477 -> 0.3499 | Helped |
| Set mask ratio to 60%, 64%, 68%, 72%, or 80% instead of 76% (ablation) | M-BEV (PETRv2) · nuScenes, one camera missing (average) | mAP: 76%: 0.3499; 60%: 0.3495; 64%: 0.3474; 68%: 0.3491; 72%: 0.3485; 80%: 0.3478 | Mixed |
Surprising findings
- The model trained with MVR also performs better than the baseline when no cameras are missing (PETRv2 NDS 0.4853 to 0.4898).
Limitations
- Small or far objects in the masked view's middle region are still hard to detect after reconstruction.
- Evaluated only on nuScenes with two BEV detectors and map segmentation; no broader benchmarks.
- Reconstructed features only approximate the original features and need fine-tuning to close the domain gap.
Experimental setup
Datasets: nuScenes
Benchmarks: nuScenes
Metrics: NDS mAP mATE mASE mAOE mAVE mAAE
Baselines: PETRv2 BEVStereo Global MVR
Builds on: Masked Autoencoders PETRv2 BEVStereo
Closest papers by meaning
- BEV-MAE: Bird’s Eye View Masked Autoencoders for Point Cloud Pre-training in Autonomous Driving Scenarios (AAAI 2024 · similarity 0.90)
- A Black-Box Evaluation Framework for Semantic Robustness in Bird’s Eye View Detection (AAAI 2025 · similarity 0.89)
- Improving Bird's Eye View Semantic Segmentation by Task Decomposition (CVPR 2024 · similarity 0.88)
- BEVDiffuser: Plug-and-Play Diffusion Model for BEV Denoising with Ground-Truth Guidance (CVPR 2025 · similarity 0.87)
- Toward Real-world BEV Perception: Depth Uncertainty Estimation via Gaussian Splatting (CVPR 2025 · similarity 0.87)
- CorrBEV: Multi-View 3D Object Detection by Correlation Learning with Multi-modal Prototypes (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.