Research map / Video / Video segmentation and tracking
Referring Segmentation in Images and Videos with Cross-Modal Self-Attention Network
Linwei Ye, Mrigank Rochan, Zhi Liu, Xiaoqin Zhang, Yang Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use word attention-aware pooling instead of max pooling (ablation) | CMSA · G-Ref val | Overall IoU: 32.96 -> 37.35 | Improved |
| use word attention-aware pooling instead of average pooling (ablation) | CMSA · G-Ref val | Overall IoU: 33.41 -> 37.35 | Improved |
| use word attention-aware pooling instead of average pooling (ablation) | CMSA · UNC+ val | Overall IoU: 32.14 -> 34.60 | Improved |
| use cross-modal self-attention instead of no attention (ablation) | CMSA · UNC val | Overall IoU: 45.63 -> 50.95 | Improved |
| use cross-modal self-attention instead of word attention only (ablation) | CMSA · UNC val | Overall IoU: 47.01 -> 50.95 | Improved |
| use cross-modal self-attention instead of pixel attention only (ablation) | CMSA · UNC val | Overall IoU: 47.84 -> 50.95 | Improved |
| use cross-modal self-attention instead of word-pixel pair attention (ablation) | CMSA · UNC val | Overall IoU: 47.57 -> 50.95 | Improved |
| keep per-word vectors (CMSA-W) instead of one LSTM sentence vector (CMSA-S) (ablation) | CMSA · UNC val | Overall IoU: 48.53 -> 50.95 | Improved |
| use gated multi-level fusion (GMLF) instead of pyramid pooling module (ablation) | CMSA · UNC val | Overall IoU: 53.54 -> 58.92 | Improved |
| use gated multi-level fusion (GMLF) instead of ConvLSTM (ablation) | CMSA · UNC val | Overall IoU: 56.56 -> 58.92 | Improved |
| use cross-level gated fusion instead of self-gated fusion (ablation) | CMSA+GMLF · UNC val | Overall IoU: 57.08 -> 58.92 | Improved |
| add cross-frame self-attention (CFSA) | Ours · A2D Sentences | Overall IoU: 59.2 -> 61.8 | Improved |
Closest papers by meaning
- Cross-Modal Self-Attention Network for Referring Image Segmentation (CVPR 2019 · similarity 0.96)
- Referring Segmentation via Encoder-Fused Cross-Modal Attention Network (TPAMI 2022 · similarity 0.93)
- Cross-Modal Progressive Comprehension for Referring Segmentation (TPAMI 2021 · similarity 0.92)
- Semantic-Assisted Object Clustering for Multi-Modal Referring Video Segmentation (TPAMI 2025 · similarity 0.91)
- Referring Image Segmentation via Cross-Modal Progressive Comprehension (CVPR 2020 · similarity 0.91)
- Language-Aware Spatial-Temporal Collaboration for Referring Video Segmentation (TPAMI 2023 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
