Research map / Vision-language and multimodal / Multimodal LLMs
DynamicVL: Benchmarking Multimodal Large Language Models for Dynamic City Understanding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces DVL-Suite (DVL-Bench and DVL-Instruct) of long-term multi-temporal NAIP imagery over 42 US cities, benchmarks 18 MLLMs on urban dynamics tasks, and builds DVLChat, a LISA-style Qwen2.5-VL 7B model with LoRA routing for VQA and referring change detection.
Problem
Existing MLLMs and remote sensing VLM datasets focus on single or bi-temporal imagery and lack pixel-precise quantitative analysis of long-term urban change, so their ability to understand multi-year city dynamics is unclear.
Approach
A long-term (2005-2023, 5-10 frames per scene) multi-temporal remote sensing benchmark with six tasks spanning pixel, regional, and scene levels, plus an instruction-tuning dataset and a LISA-style model that routes between a VQA LoRA and a segmentation LoRA on Qwen2.5-VL.
Key results
- o4-mini achieves the highest overall QA average of 34.1%, while DVLChat 7B reaches 33.3%, above its base Qwen2.5-VL 7B at 23.3%.
- On referring change detection, DVLChat reaches 29.06% IoU versus PSALM 26.93%, LISA 13.85%, and ChangeMamba 32.41%.
- DVLChat 7B achieves 3.98 on regional change captioning vs. 3.21 for Qwen2.5-VL 7B and 4.58 for o4-mini.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train Qwen2.5-VL 7B with DVL-Instruct | Qwen2.5-VL 7B (DVLChat 7B) · DVL-Bench overall QA average | accuracy (%): 23.3 -> 33.3 | Helped |
| Train Qwen2.5-VL 7B with DVL-Instruct | Qwen2.5-VL 7B (DVLChat 7B) · DVL-Bench BCA-Report | LCT: 2.49 -> 3.41 | Helped |
| Train Qwen2.5-VL 7B with DVL-Instruct | Qwen2.5-VL 7B (DVLChat 7B) · DVL-Bench regional change captioning | average score: 3.21 -> 3.98 | Helped |
| Train Qwen2.5-VL 7B with DVL-Instruct | Qwen2.5-VL 7B (DVLChat 7B) · DVL-Bench dense temporal captioning | average score: 2.85 -> 3.40 | Helped |
Surprising findings
- Scaling within model families is non-monotonic on analytical tasks: Qwen2.5-VL 32B (31.4%) outperforms 72B (29.7%), and InternVL3 14B peaks above 78B.
- A 7B domain-tuned DVLChat outperforms larger open-source models up to 78B on several tasks.
Limitations
- Current MLLMs cannot effectively use near-infrared band information, so only three optical bands are used.
- DVLChat does not leverage pixel-level segmentation data to enhance numerical quantification.
- DVLChat still trails commercial models such as o4-mini and GPT4.1 overall.
Experimental setup
Datasets: DVL-Instruct NAIP imagery environmental indicator datasets from multiple Earth observation platforms
Benchmarks: DVL-Bench
Metrics: accuracy LCT TPA CQA CRP CPA TC SA PF RC
Baselines: o4-mini GPT4.1 GPT4o Gemini 2.5 Flash TEOChat EarthDial Video-LLaVA LLaVA-OneVision InternVL3 Qwen2.5-VL LISA PSALM
Builds on: LISA Qwen2.5-VL SAM LoRA ChangeMamba
Closest papers by meaning
- UrbanFeel：A Comprehensive Benchmark for Temporal and Perceptual Understanding of City Scenes through Human Perspective (ICLR 2026 · similarity 0.91)
- XLRS-Bench: Could Your Multimodal LLMs Understand Extremely Large Ultra-High-Resolution Remote Sensing Imagery? (CVPR 2025 · similarity 0.88)
- TEOChat: A Large Vision-Language Assistant for Temporal Earth Observation Data (ICLR 2025 · similarity 0.87)
- GeoLLaVA-8K: Scaling Remote-Sensing Multimodal Large Language Models to 8K Resolution (NeurIPS 2025 · similarity 0.87)
- CityLens: Evaluating Large Vision-Language Models for Urban Socioeconomic Sensing (ICLR 2026 · similarity 0.87)
- VLRS-Bench: A Vision-Language Reasoning Benchmark for Remote Sensing (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.