Research map / Video / Video-language understanding
UniViT: Unifying Image and Video Understanding in One Vision Encoder
Feilong Tang, Xiang An, Haolin Yang, Yin Xie, Kaicheng Yang, Ming Hu, Zheng Cheng, Xingyu Zhou, Zimin Ran, Imran Razzak, Ziyong Feng, Behzad Bozorgtabar and 2 more
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes UniViT, a cluster-driven unified self-supervised vision encoder intended to capture both spatial details and temporal dynamics for image and video understanding.
Problem
Existing vision pretraining methods are biased toward either spatial modeling (e.g., CLIP) or temporal modeling (e.g., V-JEPA), limiting joint capture of spatial details and temporal dynamics.
Approach
A cluster-driven unified self-supervised learning framework that captures structured semantics across images and videos within a single vision encoder (abstract is truncated, so details beyond this are unknown).
Experimental setup
Baselines: CLIP V-JEPA
Builds on: CLIP V-JEPA
Closest papers by meaning
- Iterative Knowledge Exchange Between Deep Learning and Space-Time Spectral Clustering for Unsupervised Segmentation in Videos (TPAMI 2021 · similarity 0.88)
- Temporally-Weighted Hierarchical Clustering for Unsupervised Action Segmentation (CVPR 2021 · similarity 0.88)
- Segment Anything without Supervision (NeurIPS 2024 · similarity 0.87)
- UVU: Improving Multimodal Understanding via Vision-Language Unified Autoregressive Paradigm (CVPR 2026 · similarity 0.87)
- VONet: Unsupervised Video Object Learning With Parallel U-Net Attention and Object-wise Sequential VAE (ICLR 2024 · similarity 0.87)
- Omni-RGPT: Unifying Image and Video Region-level Understanding via Token Marks (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
