Research map / Vision-language and multimodal / Cross-modal learning
Language-Aware Vision Transformer for Referring Segmentation
Zhao Yang, Jiaqi Wang, Xubing Ye, Yansong Tang, Kai Chen, Hengshuang Zhao, Philip H. S. Torr
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add language pathway (ablation) | LAVT · RefCOCO val | oIoU: 70.78 -> 72.73 | Improved |
| add pixel-word attention module (spatial word features) instead of pooled sentence vector (ablation) | LAVT · RefCOCO val | oIoU: 71.03 -> 72.73 | Improved |
| add both language pathway and PWAM (ablation) | LAVT · RefCOCO val | oIoU: 68.82 -> 72.73 | Improved |
| use tanh instead of sigmoid in language gate (ablation) | LAVT · RefCOCO val | oIoU: 70.49 -> 72.73 | Improved |
| use InstanceNorm instead of no normalization in PWAM (ablation) | LAVT · RefCOCO val | oIoU: 70.66 -> 72.73 | Improved |
| use InstanceNorm instead of LayerNorm in PWAM (ablation) | LAVT · RefCOCO val | oIoU: 71.92 -> 72.73 | Improved |
| add language gate on F features (ablation) | LAVT · RefCOCO val | oIoU: 72.24 -> 72.73 | Improved |
| add language gate on E features (ablation) | LAVT · RefCOCO val | oIoU: 72.27 -> 72.06 | Hurt |
| add language gate on V features (ablation) | LAVT · RefCOCO val | oIoU: 72.29 -> 71.38 | Hurt |
| use PWAM instead of BCAM (ablation) | LAVT · RefCOCO val | oIoU: 70.19 -> 72.73 | Improved |
| use PWAM instead of GARAN attention (ablation) | LAVT · RefCOCO val | oIoU: 71.20 -> 72.73 | Improved |
| fuse by summation instead of concatenation (ablation) | LAVT · RefCOCO val | oIoU: 60.52 -> 72.24 | Improved |
Closest papers by meaning
- LAVT: Language-Aware Vision Transformer for Referring Image Segmentation (CVPR 2022 · similarity 0.97)
- LQMFormer: Language-aware Query Mask Transformer for Referring Image Segmentation (CVPR 2024 · similarity 0.91)
- VLT: Vision-Language Transformer and Query Generation for Referring Segmentation (TPAMI 2022 · similarity 0.91)
- Mask Grounding for Referring Image Segmentation (CVPR 2024 · similarity 0.89)
- OneRef: Unified One-tower Expression Grounding and Segmentation with Mask Referring Modeling (NeurIPS 2024 · similarity 0.88)
- Prompt-Driven Referring Image Segmentation with Instance Contrasting (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
