Research map / Time series / Foundation models for time series

Geolocation Representation from Large Language Models Are Generic Enhancers for Spatio-Temporal Learning

AAAI 2025 · Time series › Foundation models for time series · approach: LLM-based time series forecasting · 16 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes LLMGeovec, a training-free geolocation representation built by embedding OpenStreetMap-derived prompts with LLMs (LLaMa3 8B, Mistral 8x7B) and concatenating it to features for geographic prediction, long-term forecasting, and graph spatio-temporal forecasting.

Problem

Geolocation representation models are costly (street views, mobility data) and lack global coverage, and they are not used for complex spatio-temporal forecasting tasks.

Approach

Generate universal geographic prompts (instruction, reverse-geocoded address, nearby OSM places) for coordinates, embed them with a frozen LLM by averaging last-layer token states, and concatenate the resulting vector to node features of forecasting models through a small adapter, with no training of the LLM.

Key results

• On global GP, LLMGeovec (LLaMa3 8B) reaches R^2 above 0.90 on most tasks and beats Bert-whitening and GTE baselines on all 14 global tasks.
• LLMGeovec improves LTSF MSE by 2.5% to 22% average across models and datasets (e.g., iTransformer on Traffic-SD 22.01%, Global Wind 13.30%).
• On LargeST GSTF, an MLP with LLMGeovec improves MAE by 26.53% and becomes comparable to GNN models that also gain from it.

Measured effects reported in the paper

Surprising findings

• A vanilla MLP with LLMGeovec performs comparably to GNN-based STGNNs on LargeST, suggesting the embedding may partly replace graph modeling.
• Learnable node embeddings (STID) hurt MLP zero-shot transfer while LLMGeovec generalizes to new regions without adjustment.

Limitations

• Reliance on a single 8B-scale open LLM variant in most experiments due to limited computational resources; larger LLMs left to future work.
• Country and city GP comparisons rely on existing benchmarks and baseline numbers copied from prior work, with LLMGeovec losing on Crime Rate at city scale.
• Global GP results show weaker R^2 on Monthly Climate Moisture (0.55).

Experimental setup

Datasets: CHELSA WorldPop VIIRS nightlights SEDAC DHS MapillaryGCN India dataset HKGL NYC dataset Global Wind Global Temp Solar Energy Demand-SH Air Quality

Benchmarks: Multi-scale multi-topic GP benchmark (global, country, city) Global Wind Global Temp Solar Energy Demand-SH Air Quality Traffic-SD LargeST (SD, GLA, GBA) LaDe zero-shot

Metrics: MAE RMSE R^2 MSE

Baselines: Bert-whitening GTE-large GTE-qwen2 7B MapillaryGCN HKGL iTransformer TSMixer RMLP Informer DCRNN STGCN ASTGCN

Builds on: iTransformer TSMixer STID Bert-whitening Time-Series-Library

Closest papers by meaning

• Vision-Language Reasoning for Geolocalization: A Reinforcement Learning Approach (AAAI 2026 · similarity 0.88)
• Textual Supervision Enhances Geospatial Representations in Vision-Language Models (ICML 2026 · similarity 0.87)
• GeoLLM: Extracting Geospatial Knowledge from Large Language Models (ICLR 2024 · similarity 0.87)
• GeoReasoner: Geo-localization with Reasoning in Street Views using a Large Vision-Language Model (ICML 2024 · similarity 0.87)
• Recognition through Reasoning: Reinforcing Image Geo-localization with Large Vision-Language Models (NeurIPS 2025 · similarity 0.86)
• GRE Suite: Geo-localization Inference via Fine-Tuned Vision-Language Models and Enhanced Reasoning Chains (NeurIPS 2025 · similarity 0.86)