Research map / Vision-language and multimodal / Cross-modal learning
GOMAA-Geo: GOal Modality Agnostic Active Geo-localization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GOMAA-Geo, a goal-modality-agnostic active geo-localization agent that aligns aerial, ground, and text goal embeddings via CLIP-based contrastive learning, pretrains an LLM with goal-aware supervised pretraining (GASP), and trains a PPO planner, generalizing from aerial-only training to other goal modalities and disaster imagery.
Problem
Actively localizing a goal specified in one of several modalities (text, ground-level image, aerial image) using only a limited budget of sequentially observed aerial views, with zero-shot generalization across modalities and environments such as disasters.
Approach
Combine a CLIP-aligned aerial image encoder (so aerial, ground, and text goals share one embedding space) with an LLM pretrained via supervised action prediction on random trajectories (GASP) as a history encoder, then train a PPO actor-critic on top of the frozen LLM with a dense distance-based reward.
Key results
- On Masa with a 5x5 grid and budget 10, GOMAA-Geo reaches SR 0.4090 at distance 4 and 0.7854 at distance 8, with 129% to 233% relative SR gains over baselines.
- Across goal modalities on MM-GAG, GOMAA-Geo achieves comparable SR for text, ground-image, and aerial goals (e.g., 0.6766, 0.6808, 0.6638 at distance 6), despite aerial-only training.
- Zero-shot to xBD-disaster yields SR 0.4002 to 0.6942 for distances 4 to 8, a 221% to 347% improvement over baselines.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use GOMAA-Geo instead of the strongest baseline, DiT | Active geo-localization policy · Masa, C=4–8 | success ratio: 0.2011 -> 0.4090; 0.2956 -> 0.5056; 0.3567 -> 0.7168; 0.4216 -> 0.8034; 0.4559 -> 0.7854 | Helped |
| mask out the goal token (ablation) | GOMAA-Geo · Masa, C=4–8 | success ratio: 0.4090 -> 0.2913; 0.5056 -> 0.3566; 0.7168 -> 0.4912; 0.8034 -> 0.5200; 0.7854 -> 0.5478 | Hurt |
| remove the planner module (ablation) | GOMAA-Geo · Masa, C=4–8 | success ratio: 0.4090 -> 0.2331; 0.5056 -> 0.2591; 0.7168 -> 0.3121; 0.8034 -> 0.3967; 0.7854 -> 0.4051 | Hurt |
| replace GASP with masked-token autoregressive pretraining (ablation) | GOMAA-Geo · Masa, C=4–8 | success ratio: 0.4090 -> 0.4116; 0.5056 -> 0.5167; 0.7168 -> 0.6589; 0.8034 -> 0.7643; 0.7854 -> 0.7023 | Mixed |
| replace dense reward with sparse reward (ablation) | GOMAA-Geo · Masa, C=4–8 | success ratio: 0.4090 -> 0.3562; 0.5056 -> 0.4312; 0.7168 -> 0.6009; 0.8034 -> 0.7318; 0.7854 -> 0.6978 | Hurt |
| replace CLIP-MMFE with SatMAE (ablation) | GOMAA-Geo · Masa, aerial goals, C=4–8 | success ratio: 0.4090 -> 0.4329; 0.5056 -> 0.5221; 0.7168 -> 0.7016; 0.8034 -> 0.7914; 0.7854 -> 0.7992 | Mixed |
| use text, ground-image, or aerial-image goals after aerial-only training | GOMAA-Geo · MM-GAG, C=4–8 | success ratio: Text: 0.4000, 0.4978, 0.6766, 0.7702, 0.6595; Ground Image: 0.4383, 0.5150, 0.6808, 0.7489, 0.6893; Aerial Image: 0.4085 | No clear effect |
| replace uniform distance sampling with random sampling during training (ablation) | GOMAA-Geo planner · Masa and xBD-disaster, C=4–8 | success ratio: Masa: 0.4090 vs 0.4913; 0.5056 vs 0.5567; 0.7168 vs 0.6858; 0.8034 vs 0.7412; 0.7854 vs 0.6908; xBD-disaster: 0.4002 vs | Mixed |
Surprising findings
- A SatMAE-based modality-specific variant performs slightly better than GOMAA-Geo on the aerial-goal Masa setting at some distances (e.g., C=4: 0.4329 vs 0.4090).
- The LLM alone (LLM-Geo, without the planner) performs far worse than the full agent, indicating an LLM history encoder is insufficient without a learned planner.
Limitations
- Performance drops noticeably as start-to-goal distance grows and with larger grids, leaving the task far from solved.
- Results depend strongly on the choice of LLM backbone; some architectures (Mistral, Llama-2) perform poorly and the reason is not understood.
- Evaluated on a single aerial-grid simulation setup with limited search areas, so real-world UAV deployment is not tested.
Experimental setup
Datasets: Massachusetts Buildings (Masa) xBD MM-GAG
Benchmarks: Masa MM-GAG xBD-pre xBD-disaster
Metrics: success ratio
Baselines: Random PPO Decision Transformer AiRLoc LLM-Geo Mask-GOMAA RPG-GOMAA SatMAE-Geo Sparse-GOMAA
Builds on: CLIP PPO Decision Transformer AiRLoc SatMAE
Closest papers by meaning
- Vision-Language Reasoning for Geolocalization: A Reinforcement Learning Approach (AAAI 2026 · similarity 0.85)
- Game4Loc: A UAV Geo-Localization Benchmark from Game Data (AAAI 2025 · similarity 0.85)
- Weather-Robust Cross-View Geo-Localization via Prototype-Based Semantic Part Discovery (NeurIPS 2026 · similarity 0.85)
- WeatherPrompt: Multi-modality Representation Learning for All-Weather Drone Visual Geo-Localization (NeurIPS 2025 · similarity 0.84)
- Multi-Modal Aerial-Ground Cross-View Place Recognition with Neural ODEs (CVPR 2025 · similarity 0.83)
- BeliefMapNav: 3D Voxel-Based Belief Map for Zero-Shot Object Navigation (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.