Research map / Vision-language and multimodal / Multimodal LLMs

Improving Region Representation Learning from Urban Imagery with Noisy Long-Caption Supervision

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Interleaved image-text generation with MLLMs · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes UrbanLN, a CLIP-based cross-modal pre-training framework for urban region representation that handles long LLM/MLLM captions via stretched positional interpolation and mitigates caption noise with multi-MLLM captioning, consensus selection, and momentum self-distillation.

Problem

Urban region representation learning from imagery with long, noisy MLLM-generated captions: CLIP's 77-token text limit hampers fine-grained alignment, and caption noise (hallucinations, omissions, homogeneity) degrades cross-modal knowledge integration.

Approach

Information-preserved stretching interpolation extends CLIP's text positional embeddings (keeping the first 20 fixed and interpolating the rest, λ=4, max length 248); caption noise is handled at the data level by multi-MLLM captioning, SAM-based divide-and-conquer refinement, and CAPTURE-based consensus selection, and at the model level by a momentum teacher whose pseudo-targets are distilled into

Key results

• On BJ, UrbanLN+SV reaches R² 0.705 for population versus 0.619 for the strongest baseline UrbanVLP (about 13.9% relative gain), and reports average gains of 18.23% R², 7.84% RMSE and 8.32% MAE over the strongest baselines.
• On NY, UrbanLN+SI reaches R² 0.725 for population prediction (35.8% relative gain over UrbanVLP's 0.534), but UrbanLN+SI underperforms on crime (0.323 vs 0.467 for UrbanVLP).
• Ablation: removing IPSI causes an average 26.45% performance drop, the largest of the four components; removing refinement causes a 10.45% drop.

Surprising findings

• UrbanLN+SI (satellite) underperforms UrbanVLP on NY crime (R² 0.323 vs 0.467) and POI prediction, which the authors attribute to satellite imagery limitations.
• UrbanCLIP achieves the highest FPS among compared models but at lower prediction accuracy.

Limitations

• Satellite-only variant lags on NY crime and POI prediction, attributed to satellite imagery resolution and granularity limits.
• Relies on multiple large MLLMs for caption generation, which is costly as a preprocessing step (excluded from efficiency analysis).

Experimental setup

Datasets: Baidu Maps street-view and satellite imagery (BJ, SH, SZ) NY urban imagery dataset (from prior work) WorldPop population ODIAC CO2 Dianping restaurant comments

Benchmarks: BJ downstream indicators (Pop, GDP, Night, Com, CO2) SH downstream indicators SZ downstream indicators NY downstream indicators (Pop, Crime, POI)

Metrics: R^2 RMSE MAE

Baselines: ResNet-18 Tile2Vec PG-SimCLR RemoteCLIP UrbanCLIP MuseCL UrbanVLP

Builds on: CLIP UrbanCLIP UrbanVLP MoCo-style momentum distillation SAM CAPTURE

Closest papers by meaning

• Patch Matters: Training-free Fine-grained Image Caption Enhancement via Local Perception (CVPR 2025 · similarity 0.88)
• DynamicVL: Benchmarking Multimodal Large Language Models for Dynamic City Understanding (NeurIPS 2025 · similarity 0.87)
• UrbanFeel：A Comprehensive Benchmark for Temporal and Perceptual Understanding of City Scenes through Human Perspective (ICLR 2026 · similarity 0.86)
• TIPS: Text-Image Pretraining with Spatial awareness (ICLR 2025 · similarity 0.86)
• Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perception (ICML 2026 · similarity 0.86)
• Benchmarking Large Vision-Language Models on Fine-Grained Image Tasks: A Comprehensive Evaluation (ICLR 2026 · similarity 0.86)