Research map / Video / Video-language understanding
VideoOrion: Tokenizing Object Dynamics in Videos
Yicheng Feng, Yijiang Li, Wanpeng Zhang, Sipeng Zheng, Hao Luo, Zihao Yue, Zongqing Lu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add object-centric branch with object tokens (ablation) | VideoLLaMA2 · MVBench (ablation subset data) | accuracy: 41.9 -> 44.2 | Improved |
| add object-centric branch with object tokens (ablation) | VideoLLaMA2 · EgoSchema (ablation subset data) | accuracy: 41.3 -> 44.5 | Improved |
| add object-centric branch with object tokens (ablation) | VideoLLaMA2 · Perception-Test (ablation subset data) | accuracy: 43.6 -> 46.3 | Improved |
| add object-centric branch with object tokens (ablation) | VideoLLaMA2 · ActivityNet-QA (ablation subset data) | accuracy: 43.0 -> 43.3 | No clear effect |
| split video into clips with RAM++ tag changes instead of no split (ablation) | VideoOrion · Average over 5 benchmarks (ablation subset data) | average accuracy: 43.9 -> 44.9 | Improved |
| split with RAM++ instead of uniform segmentation (ablation) | VideoOrion · Average over 5 benchmarks (ablation subset data) | average accuracy: 44.7 -> 44.9 | No clear effect |
| use GroundingDINO instead of Mask2Former for object proposals (ablation) | VideoOrion · Average over 5 benchmarks (ablation subset data) | average accuracy: 43.4 -> 44.9 | Improved |
| use XMem instead of SAM2 for tracking (ablation) | VideoOrion · Average over 5 benchmarks (ablation subset data) | average accuracy: 44.4 -> 44.9 | No clear effect |
| use MLP instead of linear object projector (ablation) | VideoOrion · Average over 5 benchmarks (Table 6 subset) | average accuracy: 45.52 -> 46.73 | No clear effect |
| use MLP instead of attention object projector (ablation) | VideoOrion · Average over 5 benchmarks (Table 6 subset) | average accuracy: 45.51 -> 46.73 | Improved |
| use MLP instead of LSTM object projector (ablation) | VideoOrion · Average over 5 benchmarks (Table 6 subset) | average accuracy: 46.45 -> 46.73 | No clear effect |
| use CLIP instead of DINOv2-L for object features (ablation) | VideoOrion · Average over 5 benchmarks (Table 6 subset) | average accuracy: 43.6 -> 46.73 | Improved |
Closest papers by meaning
- Video-LaVIT: Unified Video-Language Pre-training with Decoupled Visual-Motional Tokenization (ICML 2024 · similarity 0.89)
- Streaming Long Video Understanding with Large Language Models (NeurIPS 2024 · similarity 0.89)
- LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding (NeurIPS 2024 · similarity 0.89)
- VideoRefer Suite: Advancing Spatial-Temporal Object Understanding with Video LLM (CVPR 2025 · similarity 0.89)
- ReWind: Understanding Long Videos with Instructed Learnable Memory (CVPR 2025 · similarity 0.89)
- VideoPrism: A Foundational Visual Encoder for Video Understanding (ICML 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
