Research map / Vision-language and multimodal / Multimodal LLMs
UrbanMLLM: Joint Learning of Cross-view Imagery for Urban Understanding
Xin Zhang, Tianjian Ouyang, Yu Shang, Qingmin Liao, Yong Li
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes UrbanMLLM, a unified multimodal LLM that jointly learns from satellite and street-view imagery for urban understanding.
Problem
Existing urban MLLMs rely mainly on satellite imagery, limiting capture of fine-grained street-level appearance and cross-view relationships needed for comprehensive urban understanding.
Approach
A single MLLM trained jointly on satellite and street-view imagery so it learns cross-view relationships between macroscopic spatial structure and street-level semantics.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the cross-view perceiver (ablation) | UrbanMLLM-8B · Satellite multi-scene classification | F1: 0.720 -> 0.776 | Improved |
| add the cross-view perceiver (ablation) | UrbanMLLM-8B · Satellite scene classification (single) | accuracy: 0.760 -> 0.787 | Improved |
| add the cross-view perceiver (ablation) | UrbanMLLM-8B · Satellite geo-localization | accuracy: 0.844 -> 0.871 | Improved |
| add the cross-view perceiver (ablation) | UrbanMLLM-8B · Street-view landmark recognition | accuracy: 0.753 -> 0.795 | Improved |
| add the cross-view perceiver (ablation) | UrbanMLLM-8B · Street-view geo-localization | accuracy: 0.852 -> 0.879 | Improved |
| add the cross-view perceiver (ablation) | UrbanMLLM-8B · Street-view beautiful-level prediction | accuracy (margin 2): 0.834 -> 0.827 | Hurt |
| add the cross-view perceiver (ablation) | UrbanMLLM-8B · Satellite tasks average | average performance: 3.04% drop when removed | Improved |
| add the cross-view perceiver (ablation) | UrbanMLLM-8B · Street-view tasks average | average performance: 1.54% drop when removed | Improved |
| add interleaved pre-training (ablation) | UrbanMLLM-8B · Satellite multi-scene classification | F1: 0.759 -> 0.776 | Improved |
| add interleaved pre-training (ablation) | UrbanMLLM-8B · Satellite geo-localization | accuracy: 0.846 -> 0.871 | Improved |
| add interleaved pre-training (ablation) | UrbanMLLM-8B · Satellite spatial relationship reasoning | accuracy: 0.686 -> 0.664 | Hurt |
| add interleaved pre-training (ablation) | UrbanMLLM-8B · Street-view geo-localization | accuracy: 0.882 -> 0.879 | Hurt |
Closest papers by meaning
- UrbanFeel:A Comprehensive Benchmark for Temporal and Perceptual Understanding of City Scenes through Human Perspective (ICLR 2026 · similarity 0.88)
- DynamicVL: Benchmarking Multimodal Large Language Models for Dynamic City Understanding (NeurIPS 2025 · similarity 0.88)
- Improving Region Representation Learning from Urban Imagery with Noisy Long-Caption Supervision (AAAI 2026 · similarity 0.87)
- UrBench: A Comprehensive Benchmark for Evaluating Large Multimodal Models in Multi-View Urban Scenarios (AAAI 2025 · similarity 0.87)
- Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligence (NeurIPS 2025 · similarity 0.86)
- Textual Supervision Enhances Geospatial Representations in Vision-Language Models (ICML 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
