Research map / Video / Video segmentation and tracking
Beyond Token Representations: Explicit Visual Object Grounding for Video Reasoning Segmentation
Jun Huang, zhangjunyang, Zijie Yue, Bingkun Wang, Yong Luo, Miaojing Shi
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes explicit visual object grounding for video reasoning segmentation, moving beyond segmentation tokens that are semantically monotonous and spatially ambiguous.
Problem
Video Reasoning Segmentation (VRS) must infer and segment target objects from reasoning queries in videos; existing methods compress the target into a few segmentation tokens that are semantically monotonous and spatially ambiguous, which is insufficient for mask decoding.
Approach
Replaces or augments implicit segmentation tokens with explicit visual object grounding, so the model grounds the target object visually rather than representing it only through token embeddings. Details beyond the abstract are not stated in the provided text.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add spatial-aware prompt generation and refinement (box and points) instead of segmentation tokens (ablation) | EVOG · MeViS | J&F: 45.5 -> 60.1 | Improved |
| add point set prompts (ablation) | EVOG · MeViS | J&F: 49.5 -> 60.1 | Improved |
| add bounding box prompts (ablation) | EVOG · MeViS | J&F: 57.4 -> 60.1 | Improved |
| add multi-step point refinement (R=1 to trained R=4, test R=2) (ablation) | EVOG · MeViS | J&F: 53.7 -> 60.1 | Improved |
| also apply multi-step refinement to bounding boxes (ablation) | EVOG · MeViS | J&F: 60.1 -> 60.1 | No clear effect |
| add query diversification and alignment (training only) (ablation) | EVOG · MeViS | J&F: 57.2 -> 60.1 | Improved |
| use Gaussian symmetric-KL point consistency instead of nearest-point matching (ablation) | QDA · MeViS | J&F: 57.6 -> 60.1 | Improved |
| add prompt-based keyframe selection instead of image-text similarity (ablation) | EVOG · MeViS | J&F: 57.0 -> 60.1 | Improved |
| replace SAM2 decoder with SAM3 (ablation) | EVOG · MeViS | J&F: 60.1 -> 60.6 | No clear effect |
| scale MLLM backbone up (ablation) | EVOG · MeViS | J&F: 58.5 -> 60.1 | Improved |
| increase number of predicted points K from 1 to 7 (ablation) | SPGR · MeViS | J&F: 56.0 -> 60.1 | Improved |
| add full pipeline over UniPixel on ReVOS reasoning split | EVOG vs UniPixel · ReVOS reasoning | J&F: +1.8% J&F | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- VideoSeg-R1 — VideoSeg-R1:Reasoning Video Object Segmentation via Reinforcement Learning (AAAI 2026) · better on 3 of 3 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- The Devil is in Temporal Token: High Quality Video Reasoning Segmentation (CVPR 2025 · similarity 0.91)
- One Token to Seg Them All: Language Instructed Reasoning Segmentation in Videos (NeurIPS 2024 · similarity 0.89)
- Mask Grounding for Referring Image Segmentation (CVPR 2024 · similarity 0.89)
- Motion-Grounded Video Reasoning: Understanding and Perceiving Motion at Pixel Level (CVPR 2025 · similarity 0.89)
- VGent: Visual Grounding via Modular Design for Disentangling Reasoning and Prediction (CVPR 2026 · similarity 0.89)
- GROUNDHOG: Grounding Large Language Models to Holistic Segmentation (CVPR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
