AI Paper Atlas

Research map / Video / Video-language understanding

VISA: Reasoning Video Object Segmentation via Large Language Model

Cilin Yan, Haochen Wang, Shilin Yan, Xiaolong Jiang, Yao Hu, Guoliang Kang, Weidi Xie, Efstratios Gavves

ECCV 2024 · Video › Video-language understanding · approach: Event and anomaly reasoning in video

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
remove image datasets from training (ablation)VISA · ReVOS val (reasoning)J&F: 43.0 -> 33.5Hurt
add Referring VOS datasets to training (ablation)VISA · ReVOS val (reasoning)J&F: 39.9 -> 43.0Improved
add ReVOS instruction tuning data (ablation)VISA · ReVOS val (reasoning)J&F: 39.2 -> 43.0Improved
add video VQA data to training (ablation)VISA · ReVOS val (reasoning)J&F: 41.4 -> 43.0Improved
add ReVOS instruction tuning data on referring segmentation (ablation)VISA · ReVOS val (referring)J&F: 52.9 -> 50.9No clear effect
use Text-guided Frame Sampler target frame instead of first frame (ablation)VISA · ReVOS overallJ&F: 42.6 -> 44.3Improved
add 12 reference frames with Global-Local sampling (ablation)VISA · ReVOS overallJ&F: 44.3 -> 46.9Improved
use Global-Local sampling instead of Global or Local only (ablation)VISA · ReVOS overallJ&F: 46.7 -> 46.9No clear effect
reduce visual tokens per frame from 112 to 56 (ablation)VISA · ReVOS (reasoning)J&F: 43.0 -> 39.0Hurt
increase visual tokens per frame from 112 to 256 (ablation)VISA · ReVOS (reasoning)J&F: 43.0 -> 43.2No clear effect
chain video VQA (LLaMA-VID) with referring VOS (LMPM) for reasoning queriesLMPM · ReVOS (reasoning)J&F: 18.8 -> 18.2Hurt
VISA versus LISA on refCOCO val image segmentationVISA · refCOCO valcIoU: 74.9 -> 72.4Hurt

Later papers that compared with this method

23 later accepted papers report VISA-13B as a baseline in their results tables; 23 of them report a better number on most of the results they share with it.

15 more not listed.

Earlier methods this paper compares with

Named baselines from earlier accepted papers in this paper's results tables: better than 0 of 1 on most shared results.

Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy