Research map / Large language models / Retrieval and knowledge
GeoLLM: Extracting Geospatial Knowledge from Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Fine-tunes LLMs (GPT-3.5, Llama 2, RoBERTa, GPT-2) on prompts enriched with OpenStreetMap address and nearby-place context to extract geospatial knowledge for socioeconomic and population prediction tasks.
Problem
Geospatial ML prediction usually relies on satellite imagery or other covariates that are costly or weakly predictive; the question is whether geospatial knowledge stored in LLMs can serve as covariates, and how to elicit it.
Approach
Build a prompt from coordinates plus OpenStreetMap-derived reverse-geocoded address and the 10 nearest named places with distances and directions, then fine-tune LLMs to output the label as text (binned 0.0-9.9), using the frozen-or-LoRA knowledge in the weights rather than satellite features.
Key results
- GPT-3.5 with GeoLLM reaches Pearson r^2 of 0.776 on WorldPop population density with 10,000 samples, versus 0.677 for the Nightlight baseline and 0.359 for k-NN.
- GPT-3.5 outperforms Llama 2 by 19% and RoBERTa by 51% on average, and the paper reports about a 70% improvement over prompt-based baselines.
- On asset wealth in Africa, GPT-3.5 with 10,000 samples achieves r^2 of 0.72, comparable to satellite-based methods reported at 0.56-0.71.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GeoLLM fine-tuning with the full map-data prompt rather than the strongest baseline | GPT-3.5 (GeoLLM) · WorldPop Population, 1,000 samples | Pearson's r^2: 0.623 -> 0.733 | Helped |
| Use GeoLLM fine-tuning with the full map-data prompt rather than the strongest baseline | GPT-3.5 (GeoLLM) · DHS Asset Wealth, 1,000 samples | Pearson's r^2: 0.618 -> 0.714 | Helped |
| Use GeoLLM fine-tuning with the full map-data prompt rather than the strongest baseline | GPT-3.5 (GeoLLM) · DHS Women Edu, 1,000 samples | Pearson's r^2: 0.520 -> 0.596 | Helped |
| Use GeoLLM fine-tuning with the full map-data prompt rather than the strongest baseline | GPT-3.5 (GeoLLM) · DHS Sanitation, 1,000 samples | Pearson's r^2: 0.513 -> 0.636 | Helped |
| Use GeoLLM fine-tuning with the full map-data prompt rather than the strongest baseline | GPT-3.5 (GeoLLM) · DHS Women BMI, 1,000 samples | Pearson's r^2: 0.557 -> 0.580 | Helped |
| Use GeoLLM fine-tuning with the full map-data prompt rather than the strongest baseline | GPT-3.5 (GeoLLM) · USCB Population, 1,000 samples | Pearson's r^2: 0.452 -> 0.700 | Helped |
| Use GeoLLM fine-tuning with the full map-data prompt rather than the strongest baseline | GPT-3.5 (GeoLLM) · USCB Mean Income, 1,000 samples | Pearson's r^2: 0.428 -> 0.552 | Helped |
| Use GeoLLM fine-tuning with the full map-data prompt rather than the strongest baseline | GPT-3.5 (GeoLLM) · USCB Hispanic Ratio, 1,000 samples | Pearson's r^2: 0.651 -> 0.741 | Helped |
| Use GeoLLM fine-tuning with the full map-data prompt rather than the strongest baseline | GPT-3.5 (GeoLLM) · Zillow Home Value, 1,000 samples | Pearson's r^2: 0.743 -> 0.874 | Helped |
| Add coordinates to the prompt (ablation) | GeoLLM · WorldPop Population, 1,000 samples | Pearson's r^2: 0.72 -> 0.733 (GPT-3.5); 0.63 -> 0.630 (Llama 2); 0.36 -> 0.413 (RoBERTa) | Mixed |
| Add address to the prompt (ablation) | GeoLLM · WorldPop Population, 1,000 samples | Pearson's r^2: 0.678 -> 0.733; 0.514 -> 0.630; 0.375 -> 0.413 | Helped |
| Add nearby places to the prompt (ablation) | GeoLLM · WorldPop Population, 1,000 samples | Pearson's r^2: 0.579 -> 0.733; 0.473 -> 0.630; 0.281 -> 0.413 | Helped |
Surprising findings
- Removing map data makes Llama 2 and RoBERTa ineffective, while GPT-3.5 remains more resilient.
- Unsupervised next-token training on prompts plus labels works better than supervised fine-tuning for Llama 2, and Llama 2 keeps improving even as validation loss rises.
Limitations
- GPT-3.5 is a closed model whose architecture and fine-tuning procedure are not disclosed.
- DHS coordinates are jittered by up to 2-10 km, which adds noise and limits achievable r^2.
- Preliminary evidence shows moderate performance bias toward densely populated and developed areas.
Experimental setup
Datasets: WorldPop DHS (via SustainBench) USCB ACS/Census Zillow ZHVI OpenStreetMap (Nominatim, Overpass API) VIIRS nightlights
Benchmarks: WorldPop population density DHS asset wealth DHS women education DHS sanitation DHS women BMI USCB population USCB mean income USCB Hispanic ratio Zillow home value
Metrics: Pearson r^2 mean absolute error
Baselines: k-NN XGBoost XGBoost-FT MLP-BERT Nightlight GBT GPT-2 RoBERTa 10-shot GPT-3.5 10-shot GPT-4
Builds on: QLoRA RoBERTa GPT-3.5 fine-tuning API XGBoost fastText
Closest papers by meaning
- Large Language Models are Geographically Biased (ICML 2024 · similarity 0.89)
- Geolocation Representation from Large Language Models Are Generic Enhancers for Spatio-Temporal Learning (AAAI 2025 · similarity 0.87)
- FSTLLM: Spatio-Temporal LLM for Few Shot Time Series Forecasting (ICML 2025 · similarity 0.86)
- Unifying Knowledge from Diverse Datasets to Enhance Spatial-Temporal Modeling: A Granularity-Adaptive Geographical Embedding Approach (ICML 2025 · similarity 0.86)
- GeoMMBench and GeoMMAgent: Toward Expert-Level Multimodal Intelligence in Geoscience and Remote Sensing (CVPR 2026 · similarity 0.86)
- Textual Supervision Enhances Geospatial Representations in Vision-Language Models (ICML 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.