Research map / Video / Video segmentation and tracking
Change3D: Revisiting Change Detection and Captioning from A Video Modeling Perspective
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Change3D treats bi-temporal remote sensing images plus learnable perception frames as a tiny video, using a pretrained 3D video encoder (X3D-L) instead of 2D image encoders with separate change extractors, for change detection and captioning.
Problem
Existing bi-temporal change detection and captioning methods use shared-weight 2D image encoders that are task-agnostic and need task-specific change extractors, leading to parameter distributions that under-emphasize change learning and fragmented per-task designs.
Approach
Insert learnable perception frames between the two images along the temporal axis so a video encoder's spatiotemporal attention lets perception frames directly interact with and perceive the differences; perception features then feed lightweight decoders, removing dedicated change extractors and giving one unified framework across BCD, SCD, BDA and captioning.
Key results
- On LEVIR-CD, Change3D reaches 91.82 F1 with 1.54M params and 8.29 GFLOPs, versus AMTNet's 90.76 F1 at 16.44M params.
- On xBD, Change3D achieves 79.42 F-overall with 1.60M params, outperforming PCDASNet (77.33) and ChangeOS-R101 (75.50).
- On LEVIR-CC, Change3D reports CIDEr 138.29 and BLEU-4 40.03 with 5.05M params, above SEN's 136.02 and 39.59.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Change3D with X3D-L instead of the strongest listed baseline | Change detection · LEVIR-CD | F1: 91.20 -> 91.82 | Helped |
| use Change3D with X3D-L instead of the strongest listed baseline | Change detection · WHU-CD | F1: 92.27 -> 94.56 | Helped |
| use Change3D with X3D-L instead of the strongest listed baseline | Change detection · CLCD | F1: 75.10 -> 78.03 | Helped |
| use Change3D with X3D-L instead of the strongest listed baseline | Semantic change detection · HRSCD | F1: 71.72 -> 73.29 | Helped |
| use Change3D with X3D-L instead of the strongest listed baseline | Semantic change detection · SECOND | F1: 62.63 -> 62.83 | Helped |
| use Change3D with X3D-L instead of the strongest listed baseline | Building damage assessment · xBD | Foverall1: 77.33 -> 79.42 | Helped |
| use Change3D with X3D-L instead of the strongest listed baseline | Change captioning · LEVIR-CC | CIDEr: 136.44 -> 138.29 | Helped |
| use Change3D with X3D-L instead of the strongest listed baseline | Change captioning · DUBAI-CC | CIDEr: 85.44 -> 86.19 | Helped |
| add perception frames (ablation) | Change3D · HRSCD | Fscd: 69.48 -> 73.29 | Helped |
| replace X3D-L with I3D (ablation) | Change3D · xBD | Foverall1: 79.42 -> 77.36 | Hurt |
| replace X3D-L with UniFormer-XS (ablation) | Change3D · xBD | Foverall1: 79.42 -> 78.94 | Hurt |
| use random initialization instead of pre-trained weights (ablation) | Change3D with Slow-R50 · xBD | Foverall1: 78.67 -> 74.38 | Hurt |
Surprising findings
- A generic action-recognition video encoder pretrained on Kinetics-400 transfers well to remote sensing change tasks, and even Something-Something V2 or AVA initializations beat random init by a clear margin.
- Inserting the perception frame in the middle of the bi-temporal frames (I1, IP, I2) works best, while placing it outside the pair performs worse.
Limitations
- Gains on some metrics are modest; on several datasets results are only comparable rather than best, and on CLCD and HRSCD mIoU differences are small.
- Evaluated only on remote sensing change tasks; generality to other video-like or non-remote-sensing settings is only suggested.
Experimental setup
Datasets: LEVIR-CD WHU-CD CLCD HRSCD SECOND xBD LEVIR-CC DUBAI-CC Kinetics-400
Benchmarks: LEVIR-CD WHU-CD CLCD HRSCD SECOND xBD LEVIR-CC DUBAI-CC
Metrics: F1 IoU OA SeK mIoU BLEU-N METEOR ROUGE-L CIDEr parameters
Baselines: AMTNet Bi-SRNet ChangeOS DamFormer PCDASNet SEN PromptCC RSICCformer ChangeFormer BIT
Builds on: X3D SlowFast I3D UniFormer Vision Transformer CLS token
Closest papers by meaning
- Zero-Shot Scene Change Detection (AAAI 2025 · similarity 0.88)
- Segment Any Change (NeurIPS 2024 · similarity 0.86)
- Towards Generalizable Scene Change Detection (CVPR 2025 · similarity 0.86)
- VisMin: Visual Minimal-Change Understanding (NeurIPS 2024 · similarity 0.86)
- Multi-View Pose-Agnostic Change Localization with Zero Labels (CVPR 2025 · similarity 0.86)
- Region-aware Difference Distilling with Attribute-guided Contrastive Regularization for Change Captioning (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.