Research map / Point clouds / Point cloud classification
Text to Point Cloud Localization with Multi-Level Negative Contrastive Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes multi-level negative contrastive learning (global, instance, and relation levels) for text-to-point-cloud localization on KITTI360Pose, using language to exclude mismatched locations rather than aligning features.
Problem
Language-based localization of LiDAR point clouds in city-scale maps is hard because text is ambiguous and point clouds are sparse with repeated instances; standard contrastive alignment of global features may hurt point cloud expressiveness.
Approach
Instead of pulling matched text-cell pairs together, minimize similarity of mismatched locations at global, instance, and relation levels (negative contrastive learning, weighted by a scale parameter a and d), with scene graphs built over instances and inter-instance relations via GCN.
Key results
- Top-1 submap retrieval recall on test set 0.44 vs Text2Loc 0.28 (about 57% relative gain).
- Top-1 localization recall within 5m on validation set 0.54 vs Text2Loc 0.37 (45.9% improvement claimed).
- Top-10 localization recall at 10m reaches 0.93 on test set, claimed first over 90% at that threshold.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use multi-level negative contrastive learning instead of Text2Loc | Language-based localization on KITTI360Pose · KITTI360Pose test | Submap retrieval recall (Top-1): 57% | Helped |
| Use multi-level negative contrastive learning instead of Text2Loc | Language-based localization on KITTI360Pose · KITTI360Pose validation | Localization recall (Top-1, <5m): 45.9% | Helped |
| Add instance-level features to global-level features (ablation) | MNCL · KITTI360Pose validation | Submap retrieval recall (Top-1): 0.40 -> 0.44 | Helped |
| Add relation-level features to global-level features (ablation) | MNCL · KITTI360Pose validation | Submap retrieval recall (Top-1): 0.40 -> 0.46 | Helped |
| Add both instance-level and relation-level features (ablation) | MNCL · KITTI360Pose validation | Submap retrieval recall (Top-1): 0.46 -> 0.50 | Helped |
| Replace multi-level negative contrastive loss with vanilla contrastive loss (ablation) | MNCL · KITTI360Pose validation | Submap retrieval recall (Top-1): 0.50 -> 0.45 | Hurt |
| Replace multi-level negative contrastive loss with vanilla contrastive loss (ablation) | MNCL · KITTI360Pose test | Submap retrieval recall (Top-1): 0.44 -> 0.41 | Hurt |
| Increase the number of GCN layers from zero to one (ablation) | MNCL · KITTI360Pose test | Sum of Top-1/3/5 retrieval recall rates: 162.7% to 179.6% | Helped |
| Increase the number of GCN layers beyond three (ablation) | MNCL · KITTI360Pose test | Sum of Top-1/3/5 retrieval recall rates | Hurt |
| Set the negative-loss parameter a to two (ablation) | MNCL · KITTI360Pose test | Sum of Top-1/3/5 retrieval recall rates: 185.5 | Helped |
| Increase database radius from 200m to 500m (ablation) | Ours · Scene-0009 scalability evaluation | Top-1 retrieval recall: 9.6% | Hurt |
| Increase database radius from 200m to 500m | Text2Loc · Scene-0009 scalability evaluation | Top-1 retrieval recall: 19.2% | Hurt |
Surprising findings
- Scaling the database from 200m to 500m radius degrades performance by only 9.6% versus 19.2% for Text2Loc, suggesting better scalability.
Limitations
- Experiments conducted on a single benchmark (KITTI360Pose) due to lack of other datasets.
- Fine localization keeps the Text2Loc regression architecture unchanged; no regression head improvements explored.
Experimental setup
Datasets: KITTI360Pose
Benchmarks: KITTI360Pose
Metrics: Top-k retrieval recall localization recall at 5/10/15m thresholds
Baselines: Text2Pos RET Text2Loc
Builds on: Text2Loc CLIP RCL (complementary contrastive learning) ESA (external space attention)
Closest papers by meaning
- Text2Loc: 3D Point Cloud Localization from Natural Language (CVPR 2024 · similarity 0.92)
- CMMLoc: Advancing Text-to-PointCloud Localization with Cauchy-Mixture-Model Based Framework (CVPR 2025 · similarity 0.87)
- Mechanisms of Object Localization in Vision-Language Models (CVPR 2026 · similarity 0.86)
- Pixel-Aligned Language Model (CVPR 2024 · similarity 0.85)
- LocLLM: Exploiting Generalizable Human Keypoint Localization via Large Language Model (CVPR 2024 · similarity 0.85)
- Recognition through Reasoning: Reinforcing Image Geo-localization with Large Vision-Language Models (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.