Research map / Vision-language and multimodal / Cross-modal learning
WeatherPrompt: Multi-modality Representation Learning for All-Weather Drone Visual Geo-Localization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes WeatherPrompt, a training-free chain-of-thought captioning pipeline using a large vision-language model to produce multi-weather text descriptions, plus an X-VLM-based image-text model with text-driven channel gating for all-weather drone geo-localization.
Problem
Drone visual geo-localization degrades under weather perturbations (rain, fog, snow, night), and existing methods rely on a limited set of discrete weather labels and entangled pseudo-weather features.
Approach
Generate open-set, structured weather and spatial text descriptions for drone images with chain-of-thought prompting of an off-the-shelf LVLM (no manual annotation), then fuse image and text embeddings with a text-conditioned channel gate and train with ITC, ITM, localized alignment and classification losses.
Key results
- On University-1652 drone-to-satellite, mean R@1 reaches 77.14% (+11.99 over the best listed baseline) and mean AP 80.20%.
- On SUES-200 satellite-to-drone, mean R@1 reaches 80.73% (+11.30%) and mean AP 66.12% (+14.98%) over MuSe-Net.
- On the real-world Dark+Rain+Fog video set, satellite-to-drone R@1 is 66.66% versus 38.89% for MuSe-Net.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use WeatherPrompt instead of MuSe-Net | University-1652 Drone→Satellite · University-1652 Drone→Satellite mean | R@1: 65.15 -> 77.14 (+11.99%) | Helped |
| Use WeatherPrompt instead of MuSe-Net | University-1652 Drone→Satellite · University-1652 Drone→Satellite mean | AP: 69.16 -> 80.20 (+11.04%) | Helped |
| Use WeatherPrompt instead of MuSe-Net | University-1652 Satellite→Drone · University-1652 Satellite→Drone mean | R@1: 84.68 -> 87.72 (+3.04%) | Helped |
| Use WeatherPrompt instead of MuSe-Net | University-1652 Satellite→Drone · University-1652 Satellite→Drone mean | AP: 65.75 -> 76.39 (+10.64%) | Helped |
| Use WeatherPrompt instead of MuSe-Net | SUES-200 Drone→Satellite · SUES-200 Drone→Satellite mean | R@1: 52.02 -> 61.20 (+9.18%) | Helped |
| Use WeatherPrompt instead of MuSe-Net | SUES-200 Drone→Satellite · SUES-200 Drone→Satellite mean | AP: 53.33 -> 63.26 (+9.80%) | Helped |
| Use WeatherPrompt instead of MuSe-Net | SUES-200 Satellite→Drone · SUES-200 Satellite→Drone mean | R@1: 69.43 -> 80.73 (+11.30%) | Helped |
| Use WeatherPrompt instead of MuSe-Net | SUES-200 Satellite→Drone · SUES-200 Satellite→Drone mean | AP: 51.14 -> 66.12 (+14.98%) | Helped |
| Replace concatenation or static gating with text-driven dynamic gating (ablation) | WeatherPrompt · University-1652 Drone→Satellite mean | AP: 78.90 -> 80.20; 78.45 -> 80.20 | Helped |
| Replace concatenation or static gating with text-driven dynamic gating (ablation) | WeatherPrompt · University-1652 Drone→Satellite mean | R@1: 75.73 -> 77.14; 75.26 -> 77.14 | Helped |
| Add text-based captions and supervision rather than removing all text components (ablation) | WeatherPrompt · University-1652 Drone→Satellite mean | R@1: +2.61% | Helped |
| Add text-based captions and supervision rather than removing all text components (ablation) | WeatherPrompt · University-1652 Drone→Satellite mean | AP: +1.48% | Helped |
Surprising findings
- Sample4Geo achieves the top normal-weather and fog scores on some University-1652 settings, but collapses on rain+snow and fog+snow conditions.
Limitations
- Evaluation relies on existing datasets with limited geographic and weather diversity.
- Captions from off-the-shelf vision-language models inherit biases and may contain descriptor inaccuracies (e.g., haze vs. fog).
- Single-run results without error bars or significance tests.
Experimental setup
Datasets: University-1652 SUES-200 YouTube drone-satellite videos (54 pairs, evaluation)
Benchmarks: University-1652 SUES-200 Real-world YouTube drone-satellite videos
Metrics: Recall@1 Recall@5 Recall@10 AP
Baselines: Sample4Geo MuSe-Net Safe-Net CCR LPN Zheng et al. backbone IBN-Net ResNet-101 DenseNet121 Swin-T
Builds on: X-VLM CLIP Sample4Geo imgaug
Closest papers by meaning
- Weather-Robust Cross-View Geo-Localization via Prototype-Based Semantic Part Discovery (NeurIPS 2026 · similarity 0.87)
- Weatherproofing Retrieval for Localization with Generative AI and Geometric Consistency (ICLR 2024 · similarity 0.87)
- Semantic Segmentation in Multiple Adverse Weather Conditions with Domain Knowledge Retention (AAAI 2024 · similarity 0.84)
- WeatherGFM: Learning a Weather Generalist Foundation Model via In-context Learning (ICLR 2025 · similarity 0.84)
- SkyMoE: A Vision-Language Foundation Model for Enhancing Geospatial Interpretation with Mixture of Experts (AAAI 2026 · similarity 0.84)
- GOMAA-Geo: GOal Modality Agnostic Active Geo-localization (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.