Research map / 3D vision / 3D generation
Map It Anywhere: Empowering BEV Map Prediction using Large-scale Public Datasets
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds an automated data engine (MIA) that pairs Mapillary FPV images with OpenStreetMap-rendered BEV semantic maps to create a ~1.2M-pair dataset, and trains a DINOv2-based camera-agnostic model (Mapper) that generalizes zero-shot better than prior BEV prediction baselines.
Problem
Monocular BEV map prediction from first-person images generalizes poorly because existing datasets come from autonomous-vehicle platforms that are small in geographic extent, costly to label, and limited in sensors and scenarios.
Approach
Use crowd-sourced Mapillary images and OpenStreetMap vectors, filtered and rasterized by an automated pipeline, to build a large, diverse FPV-BEV dataset; then train a camera-intrinsics-agnostic model with a DINOv2 encoder and OrienterNet-style lifting to predict gravity-aligned BEV semantic maps.
Key results
- Zero-shot Mapper on NuScenes reaches Avg. R,S 45.97 vs TIIM 47.83 (fully supervised) and SkyEye 34.02; Mapper improves road/sidewalk IoU by 33% on NuScenes and 144% on MIA-OOD over zero-shot baselines.
- On MIA-non-US, Mapper achieves Avg. R,S 37.26 vs TIIM 21.21 and SkyEye 20.87.
- Fine-tuning Mapper pretrained on MIA with 1% of NuScenes gives mean 24.54 vs TIIM trained on 1% at 18.42.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train Mapper on MIA-ID rather than NuScenes | Mapper · NuScenes val, zero-shot | Avg. R, S IoU: 47.83 -> 45.97 | Hurt |
| Train Mapper on MIA-ID rather than NuScenes | Mapper · NuScenes val, zero-shot | Avg. IoU: 33.19 -> 23.01 | Hurt |
| Train Mapper on MIA-ID instead of the in-domain KITTI360-BEV data | Mapper · KITTI360-BEV, zero-shot | Avg. R, S IoU: 58.40 -> 35.46 | Hurt |
| Train Mapper on MIA-ID instead of the in-domain KITTI360-BEV data | Mapper · KITTI360-BEV, zero-shot | Avg. IoU: 48.37 -> 24.15 | Hurt |
| Add OOD training data | Mapper · MIA-OOD, zero-shot | Avg. R, S IoU: 41.10 -> 44.52 | Helped |
| Add OOD training data | Mapper · MIA-OOD, zero-shot | Avg. IoU: 17.09 -> 20.56 | Helped |
| Pretrain Mapper on MIA, then fine-tune on 1% NuScenes data instead of training TIIM on that data | NuScenes map prediction · NuScenes, 1% fine-tuning data | Mean IoU: 18.42 -> 24.54 | Helped |
| Pretrain Mapper on MIA, then fine-tune on 10% NuScenes data instead of training TIIM on that data | NuScenes map prediction · NuScenes, 10% fine-tuning data | Mean IoU: 25.22 -> 27.12 | Helped |
| Train Mapper on MIA-ID rather than conventional AV datasets | Mapper · MIA-non-US, zero-shot | Avg. R, S IoU: 21.21 -> 37.26 | Helped |
| Train Mapper on MIA-ID rather than conventional AV datasets | Mapper · MIA-non-US, zero-shot | Avg. over classes IoU: 13.77 -> 17.66 | Helped |
| Train Mapper on MIA-ID rather than conventional AV datasets | Mapper · MIA-Rural | Avg. R, S IoU: 20.55 -> 21.04 | Helped |
Surprising findings
- On the MIA-Rural highway split, all models including Mapper fail to generalize (Avg. R,S about 21 for Mapper, 20.55 TIIM, 18.62 SkyEye).
- KITTI360-BEV ground truth lacks complete labels, so IoU measurements can be misleading for benchmarking.
Limitations
- Crowd-sourced data contains label and pose-registration noise compared to manually collected data.
- Only static map classes are captured; dynamic objects are excluded.
- Reproducibility of benchmarks is limited by incomplete labels in KITTI360-BEV and heuristic visibility masks for MIA.
Experimental setup
Datasets: MIA dataset (Mapillary + OpenStreetMap) NuScenes KITTI360-BEV
Benchmarks: NuScenes KITTI360-BEV MIA-ID MIA-OOD MIA-Rural MIA-non-US
Metrics: IoU macro-mean IoU average IoU over road and sidewalk
Baselines: TIIM SkyEye OrienterNet-style pipeline
Builds on: OrienterNet SkyEye Translating Images into Maps (TIIM) DINOv2 MapMachine
Closest papers by meaning
- VQ-Map: Bird's-Eye-View Map Layout Estimation in Tokenized Discrete Space via Vector Quantization (NeurIPS 2024 · similarity 0.88)
- Improving Bird's Eye View Semantic Segmentation by Task Decomposition (CVPR 2024 · similarity 0.87)
- MapExpert: Online HD Map Construction with Simple and Efficient Sparse Map Element Expert (AAAI 2025 · similarity 0.86)
- SDTagNet: Leveraging Text-Annotated Navigation Maps for Online HD Map Construction (NeurIPS 2025 · similarity 0.86)
- M-BEV: Masked BEV Perception for Robust Autonomous Driving (AAAI 2024 · similarity 0.85)
- MCD: Diverse Large-Scale Multi-Campus Dataset for Robot Perception (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.