Research map / Detection and segmentation / CNN features

Beyond Chamfer Distance: Granular Order-aware Evaluation Metric For Online Mapping

NeurIPS 2026 · Detection and segmentation › CNN features · approach: Domain-specific segmentation and detection · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SOSPA, an order-aware normalized polyline metric (GOSPA adapted to sequences, computed via Wagner-Fischer/GED), and DAP, a soft multi-instance metric built on P-GOSPA, evaluated on nuScenes against MapTRv2, StreamMapNet and MapTracker.

Problem

Standard online map evaluation uses Chamfer-distance-based mAP, which is insensitive to point ordering, gives coarse granularity, violates metric axioms, and cannot separate detection from geometric accuracy.

Approach

An order-aware GOSPA-style single-instance metric (SOSPA) computed as a minimum-cost trace of generalized edit distance, plus DAP, a soft P-GOSPA-based multi-instance metric that replaces hard TP/FP thresholding with confidence-weighted localization and detection terms.

Key results

• On nuScenes R60, mDAP ranks MapTracker (0.822) ahead of StreamMapNet (0.866) and MapTRv2 (0.893), matching the mAP ranking.
• Detection error dominates localization error by a factor of two to four for all methods on both evaluation ranges.
• On R100, FD-AP is near zero for all methods (mAP 0.029, 0.031, 0.091), whereas DAP stays informative (mDAP 0.884-0.936).

Surprising findings

• CD-based mAP gives a perfect score to predictions shifted diagonally by 1.0 m with one misordered instance, while DAP penalizes them.
• Fixed-N=200 point resampling gives similar metric values to 0.5 m equidistant sampling but takes roughly 4-6x longer to evaluate.

Limitations

• Metric focuses on geometry and does not capture instance-level topology such as lane connectivity.
• Only one-to-one assignment; does not handle fragmentation or merging of instances.
• Evaluated only on nuScenes with three methods.

Experimental setup

Datasets: nuScenes

Benchmarks: nuScenes

Metrics: mAP AP mDAP SOSPA DAP localization error DAP detection error runtime

Baselines: MapTRv2 StreamMapNet MapTracker Chamfer-distance mAP Frechet-distance AP

Builds on: GOSPA P-GOSPA generalized edit distance Wagner-Fischer algorithm mAP

Closest papers by meaning

• Producing and Leveraging Online Map Uncertainty in Trajectory Prediction (CVPR 2024 · similarity 0.86)
• SDTagNet: Leveraging Text-Annotated Navigation Maps for Online HD Map Construction (NeurIPS 2025 · similarity 0.86)
• MapExpert: Online HD Map Construction with Simple and Efficient Sparse Map Element Expert (AAAI 2025 · similarity 0.85)
• GeoRanker: Distance-Aware Ranking for Worldwide Image Geolocalization (NeurIPS 2025 · similarity 0.84)
• The Road Ahead in Autonomous Driving: The KITScenes Multimodal Dataset (NeurIPS 2026 · similarity 0.84)
• Unveiling the Hidden: Online Vectorized HD Map Construction with Clip-Level Token Interaction and Propagation (NeurIPS 2024 · similarity 0.84)