Research map / Vision-language and multimodal / Multimodal LLMs

TimeSpot: Benchmarking Geo-Temporal Understanding in Vision–Language Models in Real-World Settings

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Spatial reasoning in VLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces TimeSpot, a 1,455-image ground-level benchmark for joint geo-temporal prediction (season, month, time, daylight phase, plus geographic fields) across VLMs, finding low temporal accuracy and geo-temporal inconsistencies, with a small SFT diagnostic.

Problem

Existing geolocation benchmarks focus on spatial inference and ignore when a photo was taken; VLMs lack evaluation for joint temporal and geographic reasoning from subtle ground-level visual cues with structured, consistency-checked outputs.

Approach

A benchmark with a nine-field geo-temporal schema (four temporal, five geographic fields) whose labels are derived programmatically from metadata, ephemerides, and Köppen-Geiger maps, with human verification, plus cross-field consistency checks, calibration (ECE, RC-AUC), and a small LoRA SFT diagnostic.

Key results

• Gemini-2.5-Flash achieves 77.25% country accuracy and 917.61 km mean distance, while the best time-of-day accuracy is 33.74% (GLM-4.1V-9B-Thinking).
• Expert humans reach 57.89% time-within-1h accuracy and 1:36 time MAE, versus best VLM time MAE of 3:36.
• SFT of Qwen2.5-VL-3B raises country accuracy from 14.20% to 19.24% and time accuracy from 20.27% to 24.79%, with cross-task interference.

Surprising findings

• Country-tuned and time-tuned single-task SFT each reduce accuracy on the other task, indicating gradient interference under shared LoRA parameters.
• Autumn season prediction collapses across all evaluated models.

Limitations

• Small benchmark of 1,455 images with uneven country distribution (USA 196, many countries sparse).
• Reliance on Gemini-2.5-Flash as judge; validation described only qualitatively.
• SFT experiments are a limited diagnostic on one 3B model using a 40% split.

Experimental setup

Datasets: TimeSpot

Benchmarks: TimeSpot

Metrics: accuracy MAE geodesic distance expected calibration error risk-coverage AUC

Baselines: GPT-4o-mini GPT-5-mini Gemini-2.5-Flash Qwen-VL2.5-72B-Instruct InternVL3-78B GLM-4.5V-106B-MoE o4-mini Gemini-2.5-Flash-Thinking

Builds on: Köppen-Geiger climate classification LoRA

Closest papers by meaning

• GTR-Bench: Evaluating Geo-Temporal Reasoning in Vision-Language Models (ICLR 2026 · similarity 0.92)
• Recognition through Reasoning: Reinforcing Image Geo-localization with Large Vision-Language Models (NeurIPS 2025 · similarity 0.90)
• GRE Suite: Geo-localization Inference via Fine-Tuned Vision-Language Models and Enhanced Reasoning Chains (NeurIPS 2025 · similarity 0.90)
• ERGeoBench: A Comprehensive Benchmark for Embodied Reasoning and Geo-localization in Multimodal Large Language Models (ICML 2026 · similarity 0.89)
• GeoReasoner: Geo-localization with Reasoning in Street Views using a Large Vision-Language Model (ICML 2024 · similarity 0.89)
• Vision-Language Reasoning for Geolocalization: A Reinforcement Learning Approach (AAAI 2026 · similarity 0.88)