Research map / Vision-language and multimodal / Multimodal LLMs
Recognition through Reasoning: Reinforcing Image Geo-localization with Large Vision-Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds MP16-Reason, a 33K-sample social-media geo-localization dataset with distilled reasoning from multiple VLMs, and trains GLOBE (Qwen2.5-VL-7B) with GRPO using localizability, visual-grounding, and geo-accuracy rewards.
Problem
Image geo-localization with LVLMs suffers from street-view-dominated reasoning datasets with limited scene diversity, and SFT on instruction data yields only marginal gains in grounded reasoning.
Approach
Distill localizability judgments, reasoning traces, and locations from strong VLMs (Qwen2.5-VL-72B, InternVL3-78B, GeoCLIP) over diverse MP-16 social-media images, filter via multi-model self-verification and segmentation-based visual grounding, then train with GRPO using three task-specific rewards (localizability, visual grounding consistency, country/city accuracy).
Key results
- GLOBE-7B reaches 17.99% at 1km, 62.85% at 25km, and 86.68% at 750km on MP16-Reason-Test, above Qwen2.5-VL-72B (17.52, 59.30, 84.06).
- On IM2GPS3K, GLOBE-7B achieves 40.18% at 25km versus 35.77% for Qwen2.5-VL-72B, though it trails Doubao1.5-VL and GPT-4.1 at most thresholds.
- GRPO improves InternVL3-8B city-level accuracy from 44.17% (baseline) to 60.09% and Qwen2.5-VL-7B from 56.76% (SFT) to 62.85%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fine-tune Qwen2.5-VL-7B with GLOBE's reasoning-oriented data and GRPO (ablation) | Qwen2.5-VL-7B · MP16-Reason-Test | accuracy within 25km (%): 52.72 -> 62.85 | Helped |
| fine-tune Qwen2.5-VL-7B with GRPO rather than SFT (ablation) | Qwen2.5-VL-7B · MP16-Reason-Test | accuracy within 25km (%): 56.76 -> 62.85 | Helped |
| add the Localizability reward (ablation) | GLOBE · MP16-Reason-Test | accuracy within 25km (%): 59.58 -> 62.85 | Helped |
| add the Visual Grounding Consistency reward (ablation) | GLOBE · MP16-Reason-Test | accuracy within 25km (%): 59.83 -> 62.85 | Helped |
| add the Geo-localization Accuracy reward (ablation) | GLOBE · MP16-Reason-Test | accuracy within 25km (%): 59.53 -> 62.85 | Helped |
| add Localizability and Geo-localization Accuracy rewards (ablation) | GLOBE · MP16-Reason-Test | accuracy within 25km (%): 59.36 -> 62.85 | Helped |
| add Localizability and Visual Grounding Consistency rewards (ablation) | GLOBE · MP16-Reason-Test | accuracy within 25km (%): 59.24 -> 62.85 | Helped |
| use full multi-source validation instead of random sampling (ablation) | MP16-Reason training data with Qwen2.5-VL-7B + GRPO · MP16-Reason-Test | accuracy within 25km (%): 59.22 -> 62.85 | Helped |
| use full multi-source validation instead of single-source validation (ablation) | MP16-Reason training data with Qwen2.5-VL-7B + GRPO · MP16-Reason-Test | accuracy within 25km (%): 59.45 -> 62.85 | Helped |
| use GRPO instead of SFT (ablation) | InternVL3-8B · MP16-Reason-Test | accuracy within 25km (%): 44.68 -> 60.09 | Helped |
| use GRPO instead of SFT (ablation) | Qwen2.5-VL-7B · MP16-Reason-Test | accuracy within 25km (%): 56.76 -> 62.85 | Helped |
| set reward weights to (0.2, 0.5, 1.0) instead of (1.0, 0.5, 0.2) (ablation) | GLOBE GRPO · MP16-Reason-Test | accuracy within 25km (%): 59.96 -> 62.85 | Helped |
Surprising findings
- GLOBE-7B trained on 33K samples surpasses the 72B model used to generate its distillation labels.
- SFT on the reasoning data gives only marginal gains for InternVL3-8B (e.g., 44.17% to 44.68% at 25km), while GRPO yields large gains.
Limitations
- Reasoning-based predictions lose accuracy at fine-grained, coordinate-level localization.
- Still behind closed-source systems such as GPT-4.1 and Doubao1.5-VL.
- Dataset bias: correct reasoning can be pulled toward locations overrepresented in training data.
Experimental setup
Datasets: MP-16 MP16-Reason-Train IM2GPS3K OSV-5M
Benchmarks: MP16-Reason-Test IM2GPS3K OSV-5M (mini-3K)
Metrics: accuracy at 1km 25km 200km 750km 2500km
Baselines: GeoCLIP ISNs Qwen2.5-VL-7B Qwen2.5-VL-72B InternVL3-78B GeoReasoner-7B GPT-4.1 Doubao1.5-VL
Builds on: GRPO MP-16 Qwen2.5-VL GeoCLIP PPO
Closest papers by meaning
- GRE Suite: Geo-localization Inference via Fine-Tuned Vision-Language Models and Enhanced Reasoning Chains (NeurIPS 2025 · similarity 0.94)
- Vision-Language Reasoning for Geolocalization: A Reinforcement Learning Approach (AAAI 2026 · similarity 0.93)
- GeoReasoner: Geo-localization with Reasoning in Street Views using a Large Vision-Language Model (ICML 2024 · similarity 0.93)
- ERGeoBench: A Comprehensive Benchmark for Embodied Reasoning and Geo-localization in Multimodal Large Language Models (ICML 2026 · similarity 0.90)
- TimeSpot: Benchmarking Geo-Temporal Understanding in Vision–Language Models in Real-World Settings (ICML 2026 · similarity 0.90)
- Textual Supervision Enhances Geospatial Representations in Vision-Language Models (ICML 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.