Research map / Vision-language and multimodal / Human and embodied modeling
LAVT: Language-Aware Vision Transformer for Referring Image Segmentation
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| integrate language into intermediate vision Transformer stages with LAVT | Swin-B + BERT · RefCOCO val | oIoU: 65.65 -> 72.73 | Helped |
| integrate language into intermediate vision Transformer stages with LAVT | Swin-B + BERT · RefCOCO+ val | oIoU: 55.50 -> 62.14 | Helped |
| integrate language into intermediate vision Transformer stages with LAVT | Swin-B + BERT · G-Ref val (UMD) | oIoU: 52.99 -> 61.24 | Helped |
| add the language pathway as a residual stream (ablation) | LAVT · RefCOCO val | oIoU: 70.78 -> 72.73 | Helped |
| use pixel-word attention instead of a globally pooled sentence feature (ablation) | LAVT · RefCOCO val | oIoU: 71.03 -> 72.73 | Helped |
| use tanh rather than sigmoid as the language gate activation (ablation) | LAVT · RefCOCO val | oIoU: 70.49 -> 72.73 | Helped |
| use InstanceNorm in PWAM projections (ablation) | LAVT · RefCOCO val | oIoU: 71.92 -> 72.73; 71.59 -> 72.73; 70.66 -> 72.73 | Helped |
| classify from F features rather than E features (ablation) | LAVT with language gate · RefCOCO val | oIoU: 72.06 -> 72.73 | Helped |
| classify from F features rather than V features (ablation) | LAVT with language gate · RefCOCO val | oIoU: 71.38 -> 72.73 | Helped |
| add language gate while classifying from E features (ablation) | LAVT · RefCOCO val | oIoU: 72.27 -> 72.06 | Hurt |
| add language gate while classifying from V features (ablation) | LAVT · RefCOCO val | oIoU: 72.29 -> 71.38 | Hurt |
| replace PWAM with BCAM or GA (ablation) | LAVT · RefCOCO val | oIoU: 70.19; 71.20 vs. 72.73 | Hurt |
Closest papers by meaning
- LQMFormer: Language-aware Query Mask Transformer for Referring Image Segmentation (CVPR 2024 · similarity 0.91)
- Mask Grounding for Referring Image Segmentation (CVPR 2024 · similarity 0.89)
- UniPixel: Unified Object Referring and Segmentation for Pixel-Level Visual Reasoning (NeurIPS 2025 · similarity 0.87)
- Prompt-Driven Referring Image Segmentation with Instance Contrasting (CVPR 2024 · similarity 0.87)
- OneRef: Unified One-tower Expression Grounding and Segmentation with Mask Referring Modeling (NeurIPS 2024 · similarity 0.87)
- GSVA: Generalized Segmentation via Multimodal Large Language Models (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.