Research map / 3D vision / 3D generation

The Road Ahead in Autonomous Driving: The KITScenes Multimodal Dataset

NeurIPS 2026 · 3D vision › 3D generation · approach: 3D perception for autonomous driving · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces KITScenes Multimodal, a European robotaxi-sensor driving dataset with high-resolution cameras, 400m+ lidar, 4D radar, and complete 3D Lanelet2 HD maps, plus four spatial-learning benchmarks (online HD map construction, long-range depth, novel view synthesis, end-to-end driving).

Problem

Existing autonomous driving datasets lack high-fidelity long-range sensing, complete topology-aware HD maps with 3D regulatory elements, and European urban geographic diversity needed for spatial learning.

Approach

A fully synchronized, high-resolution, long-range sensor suite paired with reprojection-accurate, topologically complete Lanelet2 HD maps (including 3D traffic lights, signs, poles) validated in Autoware, and four benchmarks that expose gaps in current methods.

Key results

• MapQR-Topo reaches 16.4 AP_Topo on the complete Lanelet2 formulation, while MapTRv2 and SDTagNet obtain large drops on the richer task.
• MapAnything ranks first overall in monocular depth (AbsRel 0.156) but degrades to 0.772 AbsRel beyond 200 m, where UniDAC is strongest (AbsRel 0.540, delta1 1.78).
• Traffic sign recall for ReconDrive drops by over 80% relative at a 3 m lateral shift, and upsampling to sensor resolution causes a 27.8% relative recall drop.

Surprising findings

• The overall depth ranking is inverted at long range: the overall best model (MapAnything) is worst beyond 200 m, while the overall worst (UniDAC) is best there.
• Epona, a diffusion world model trained on nuPlan, shows the smallest domain gap among the zero-shot end-to-end baselines.

Limitations

• No 3D bounding boxes, tracks, or instance segmentation for dynamic agents in the current release.
• Dataset is small (5.7 hours) compared with large-scale corpora such as nuPlan Sensor and Nvidia PhysicalAI AV.
• End-to-end benchmark is open-loop only; closed-loop evaluation is left to future work.

Experimental setup

Datasets: KITScenes Multimodal

Benchmarks: KITScenes Multimodal

Metrics: AP topology AP AbsRel delta1 traffic sign recall ADE FDE drivable-surface survival collision-free rate centerline distance

Baselines: MapTRv2 SDTagNet UniDAC Depth Anything 3 MapAnything ReconDrive UniAD DMAD SSR Epona

Builds on: MapQR Lanelet2 Autoware KISS-SLAM OpenLane-V2

Closest papers by meaning

• Multiagent Multitraversal Multimodal Self-Driving: Open MARS Dataset (CVPR 2024 · similarity 0.88)
• Autonomous Driving Research Requires a Community-Driven Data Paradigm (NeurIPS 2026 · similarity 0.88)
• MCD: Diverse Large-Scale Multi-Campus Dataset for Robot Perception (CVPR 2024 · similarity 0.87)
• MAPLM: A Real-World Large-Scale Vision-Language Benchmark for Map and Traffic Scene Understanding (CVPR 2024 · similarity 0.87)
• UrbanIng-V2X: A Large-Scale Multi-Vehicle, Multi-Infrastructure Dataset Across Multiple Intersections for Cooperative Perception (NeurIPS 2025 · similarity 0.87)
• Generating Multimodal Driving Scenes via Next-Scene Prediction (CVPR 2025 · similarity 0.87)