Research map / Video / Video-language understanding
X2SAM: Any Segmentation in Images and Videos
Hao Wang, Limeng Qiao, Chi Zhang, Lin Ma, Guanglu Wan, Xiangyuan Lan, Xiaodan Liang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add zero-initialized token-to-image attention to mask decoder (ablation) | X2SAM decoder · V-Ref. YT21 | J&F: 53.6 -> 60.8 | Improved |
| add randomly initialized token-to-image attention (ablation) | X2SAM decoder · I-Ref. RefCOCO/+/g | cIoU: 82.9/78.0/79.5 -> 82.5/77.2/79.4 | Hurt |
| use zero init instead of random init for T2I attention (ablation) | X2SAM decoder · V-Ref. YT21 | J&F: 58.2 -> 60.8 | Improved |
| add single-scale mask memory (ablation) | X2SAM · V-Ref. YT21 | J&F: 53.6 -> 52.5 | No clear effect |
| add mask guidance to memory (ablation) | X2SAM memory · V-Ref. YT21 | J&F: 53.6 -> 63.3 | Improved |
| add multi-scale memory design (ablation) | X2SAM memory · V-Rea. ReVOS All | J&F: 36.5 -> 53.5 | Improved |
| increase memory size from 1 to 6 (ablation) | X2SAM · V-OV YT-VIS21 | mAP: 58.5 -> 60.2 | Improved |
| increase memory size to 8 (ablation) | X2SAM · V-Rea. ReVOS All | J&F: 57.5 -> 53.5 | Hurt |
| use unified joint training instead of simple joint training (ablation) | X2SAM · training cost | GPU hours: 5.2K -> 3.3K | Improved |
| use unified joint training instead of separate training (ablation) | X2SAM · V-Gen. mIoU | mIoU: 61.1 -> 64.7 | Improved |
| extend image model to video with mask memory (X2SAM vs X-SAM) | X2SAM · I-Gen./I-VGD image tasks | PQ/AP: I-OV 20.9 -> 31.2 PQ; I-Gen. 54.7 -> 54.1 PQ; I-VGD COCO point 47.9 -> 45.9 | Mixed |
| X2SAM vs video MLLM generalist on grounded conversation | X2SAM · V-GCG | mIoU: 54.3 -> 75.8 | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- VideoGLaMM — VideoGLaMM : A Large Multimodal Model for Pixel-Level Visual Grounding in Videos (CVPR 2025) · better on 1 of 1 shared result
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- ViCaS: A Dataset for Combining Holistic and Pixel-level Video Understanding using Captions with Grounded Segmentation (CVPR 2025 · similarity 0.91)
- One Token to Seg Them All: Language Instructed Reasoning Segmentation in Videos (NeurIPS 2024 · similarity 0.89)
- ARGenSeg: Image Segmentation with Autoregressive Image Generation Model (NeurIPS 2025 · similarity 0.88)
- SAM 2: Segment Anything in Images and Videos (ICLR 2025 · similarity 0.88)
- Segment Anyword: Mask Prompt Inversion for Open-Set Grounded Segmentation (ICML 2025 · similarity 0.88)
- Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understanding (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
