Research map / 3D vision / 3D generation
Multiagent Multitraversal Multimodal Self-Driving: Open MARS Dataset
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces the Open MARS dataset of multiagent, multitraversal, multimodal (LiDAR + surround RGB) autonomous driving recordings from May Mobility's fleet, with benchmarks for visual place recognition and neural reconstruction.
Problem
Existing autonomous driving datasets come from single-vehicle one-time traversals, lacking multiagent interactions and repeated traversals of the same location needed for collaborative perception and retrospective memory.
Approach
A real-world driving dataset from a four-vehicle fleet that combines synchronized multi-vehicle encounters with hundreds of repeated traversals of the same locations, with surround-view cameras and LiDAR, and benchmarks on place recognition and neural reconstruction.
Key results
- On multitraversal VPR, MixVPR achieves the best Recall@1 of 71.73, versus 63.51 for NetVLAD and 53.33 for ViT.
- On multiagent VPR, CoVPR reaches Recall@1 of 92.27 versus 91.85 for single-agent NetVLAD.
- On single-traversal reconstruction, EmerNeRF gets the best PSNR (29.63) while PVG gets the best SSIM (0.900) and LPIPS (0.197).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MixVPR instead of NetVLAD | VPR · MARS multitraversal | Recall @1: 63.51 -> 71.73 | Helped |
| Use CoVPR instead of NetVLAD | VPR · MARS multiagent | Recall @1: 91.85 -> 92.27 | Helped |
| Use PointNetVLAD instead of NetVLAD | VPR · MARS multitraversal | Recall @1: 63.51 -> 66.45 | Helped |
| Use EmerNeRF instead of iNGP | Neural reconstruction · MARS single-traversal | PSNR: 28.66 -> 29.63 | Helped |
| Use PVG instead of iNGP | Neural reconstruction · MARS single-traversal | SSIM: 0.821 -> 0.900 | Helped |
| Use PVG instead of iNGP | Neural reconstruction · MARS single-traversal | LPIPS: 0.256 -> 0.197 | Helped |
| Use RobustNeRF instead of iNGP | Neural reconstruction · MARS multitraversal | PSNR: 26.04 -> 16.17 | Hurt |
| Use SegNeRF instead of iNGP | Neural reconstruction · MARS multitraversal | PSNR: 26.04 -> 24.44 | Hurt |
| Use SegNeRF instead of iNGP | Neural reconstruction · MARS multitraversal | LPIPS: 0.346 -> 0.358 | Hurt |
Surprising findings
- RobustNeRF performs poorly on multitraversal reconstruction (PSNR 16.17) despite being designed to remove dynamic distractors, while iNGP scores highest on similarity metrics.
Limitations
- Benchmarks cover only place recognition and neural reconstruction; no online perception annotations are provided yet.
- Data comes from a single operator's fixed-route shuttle service in one U.S. geographic area.
- Multiagent subset is small (53 scenes, mostly two vehicles).
Experimental setup
Datasets: Open MARS multitraversal subset Open MARS multiagent subset ImageNet1K Cityscapes
Benchmarks: Open MARS multitraversal VPR Open MARS multiagent VPR Open MARS single-traversal reconstruction Open MARS multitraversal reconstruction
Metrics: Recall@1 Recall@5 Recall@10 PSNR SSIM LPIPS
Baselines: NetVLAD MixVPR GeM Plain ViT PointNetVLAD CoVPR iNGP 3DGS EmerNeRF PVG RobustNeRF SegNeRF
Builds on: NetVLAD EmerNeRF PVG CoVPR MixVPR
Closest papers by meaning
- The Road Ahead in Autonomous Driving: The KITScenes Multimodal Dataset (NeurIPS 2026 · similarity 0.88)
- Autonomous Driving Research Requires a Community-Driven Data Paradigm (NeurIPS 2026 · similarity 0.86)
- UrbanIng-V2X: A Large-Scale Multi-Vehicle, Multi-Infrastructure Dataset Across Multiple Intersections for Cooperative Perception (NeurIPS 2025 · similarity 0.85)
- Transfer Your Perspective: Controllable 3D Generation from Any Viewpoint in a Driving Scene (CVPR 2025 · similarity 0.85)
- STAMP: Scalable Task- And Model-agnostic Collaborative Perception (ICLR 2025 · similarity 0.85)
- DriveAgent-R1: Advancing VLM-based Autonomous Driving with Active Perception and Hybrid Thinking (ICLR 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.