Research map / Detection and segmentation / Open-set and few-shot detection
The STVchrono Dataset: Towards Continuous Change Recognition in Time
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces STVchrono, a real-world street-view dataset of 71,900 photos from 50 cities over 18 years for continuous change captioning (pairs and sequences) and change-aware sequential instance segmentation, and benchmarks captioning methods, multimodal LLMs and video segmenters on it.
Problem
Existing change understanding datasets focus on discrete changes between two images, often synthetic, and do not capture continuous, gradual real-world changes over long time spans.
Approach
A benchmark of Google Street View photos taken at the same locations over 18 years, with human-written change captions for image pairs and image sequences (2-6 images) plus instance masks tracked across five-image sequences, so that continuous change and trends can be evaluated.
Key results
- On image-pair captioning, CLIP4IDC was the best non-LLM baseline (BLEU4 28.5, CIDEr 69.5, GPT4 32.4), while human performance reached 21.2 BLEU4, 50.8 CIDEr, 40.0 GPT4 with human rating accuracy 94.4 vs. 47.8 for CLIP4IDC.
- On image-sequence captioning, BLIP2+GPT4 had the highest GPT4 score among methods, while OpenFlamingo reached 34.4 human accuracy vs. 89.8 for humans.
- For change-aware sequential instance segmentation, CTVIS with SwinT-L achieved the best AP of 7.08 (AP50 10.42), indicating the task remains hard.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use multimodal LLM-based captioning rather than conventional captioning methods | STVchrono image pair captioning · STVchrono image pair | GPT4 | Mixed |
| Use multimodal LLM-based captioning rather than conventional captioning methods | STVchrono image sequence captioning · STVchrono image sequence | GPT4 | Mixed |
| Increase image-sequence length | STVchrono captioning models · STVchrono image sequences of 3 to 6 images | BLEU4 | Hurt |
| Increase image-sequence length | STVchrono captioning models · STVchrono image sequences of 3 to 6 images | GPT4 | No clear effect |
| Use a larger SwinT-L backbone instead of smaller backbones (ablation) | Mask2Former sequential instance segmentation · STVchrono sequential instance segmentation | AP | No clear effect |
| Use a larger SwinT-L backbone instead of ResNet50 (ablation) | CTVIS sequential instance segmentation · STVchrono sequential instance segmentation | AP: 5.86 -> 7.08 | Helped |
Surprising findings
- Multimodal LLM methods (OpenFlamingo, BLIP2+GPT4) score lower on BLEU4 and CIDEr but higher on GPT4 score and human ratings than trained captioning models.
- Larger backbones (SwinT-L) did not significantly improve instance segmentation scores over smaller ones.
Limitations
- Uneven city data distribution.
- Restricted range of changes associated with weather and time of day.
- Change description and region detection are treated separately rather than as one integrated method.
Experimental setup
Datasets: STVchrono
Benchmarks: STVchrono
Metrics: BLEU4 CIDEr GPT4 score human rating accuracy human rating coverage AP AP50 AP75
Baselines: DUDA MCCFormers-D MCCFormers-S CLIP4IDC VARD-Trans OpenFlamingo BLIP2 + GPT4 Mask2Former CTVIS
Builds on: Mask2Former CTVIS CLIP4IDC MCCFormers
Closest papers by meaning
- The Change You Want To Detect: Semantic Change Detection In Earth Observation With Hybrid Data Generationf (CVPR 2025 · similarity 0.87)
- DynamicVL: Benchmarking Multimodal Large Language Models for Dynamic City Understanding (NeurIPS 2025 · similarity 0.87)
- RSCC: A Large-Scale Remote Sensing Change Caption Dataset for Disaster Events (NeurIPS 2025 · similarity 0.85)
- Towards Generalizable Scene Change Detection (CVPR 2025 · similarity 0.85)
- CityPulse: Fine-Grained Assessment of Urban Change with Street View Time Series (AAAI 2024 · similarity 0.85)
- Change3D: Revisiting Change Detection and Captioning from A Video Modeling Perspective (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.