Research map / Vision-language and multimodal / Multimodal LLMs

UrbanFeel：A Comprehensive Benchmark for Temporal and Perceptual Understanding of City Scenes through Human Perspective

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Benchmarks and analysis of MLLMs

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces UrbanFeel, a 14.3K-question benchmark of single-view and panoramic multi-temporal street-view images from 11 cities for evaluating 20 MLLMs on static scene perception, temporal change understanding, and subjective urban perception.

Problem

Existing urban MLLM benchmarks are limited to static snapshots and objective tasks, lacking evaluation of temporal urban evolution and alignment with human subjective perception of city scenes.

Approach

A human-centric benchmark that combines multi-temporal, multi-view street-view imagery with subjective perception questions (beautiful, safe, wealthy, lively) and temporal sorting/change tasks, built through a hybrid pipeline of spatial clustering, rule-based generation, model-assisted prompting and manual annotation.

Key results

• Gemini-2.5 Pro achieves the best overall accuracy of 65.9%, narrowing the average gap to human experts (67.4%) to 1.5%.
• Most models score below 10% on the TSR temporal sorting task; Gemini-2.5 Pro lags human performance by 17.9% there.
• On average, single-view inputs yield 11.7% higher accuracy than panoramic inputs.

Surprising findings

• In the panoramic PCR task, MLLMs outperform human evaluators.
• Assigning city identity labels shifts subjective judgments; GPT-4o's wealth ratings for 'Paris' and 'Milan' drop below those for 'Cape Town'.

Limitations

• Performance on open-ended and open-source models is weaker, and city identity labels introduce bias in subjective judgments.
• Subjective labels rely on a small human annotator pool with geography backgrounds.

Experimental setup

Datasets: Mapillary street-view imagery Google Street View API imagery

Benchmarks: UrbanFeel

Metrics: accuracy semantic similarity

Baselines: GPT-4o Gemini-2.5 Pro DeepSeek-VL2 InternVL3 LLaVA-1.6 Qwen2.5-VL Phi-4 Gemma-3 Aria Mistral-Small-3.1-24B Idefics3-8B Human

Builds on: MMMU UrBench

Closest papers by meaning

• DynamicVL: Benchmarking Multimodal Large Language Models for Dynamic City Understanding (NeurIPS 2025 · similarity 0.91)
• CityLens: Evaluating Large Vision-Language Models for Urban Socioeconomic Sensing (ICLR 2026 · similarity 0.89)
• UrBench: A Comprehensive Benchmark for Evaluating Large Multimodal Models in Multi-View Urban Scenarios (AAAI 2025 · similarity 0.89)
• UrbanDataLayer: A Unified Data Pipeline for Urban Science (NeurIPS 2024 · similarity 0.87)
• CityPulse: Fine-Grained Assessment of Urban Change with Street View Time Series (AAAI 2024 · similarity 0.87)
• CitySeeker: How Do VLMs Explore Embodied Urban Navigation with Implicit Human Needs? (ICLR 2026 · similarity 0.86)