Research map / Video / Video segmentation and tracking
Generative Video Transformer: Can Objects be the Words?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use object-centric latents instead of a convolutional feature grid (ablation) | OCVT · Bouncing Balls Mod1 next-step prediction | color change accuracy: 88.31 -> 89.61 | Helped |
| Use object-centric latents instead of a convolutional feature grid (ablation) | OCVT · Bouncing Balls Mod2 next-step prediction | color change accuracy: 82.83 -> 88.18 | Helped |
| Use object-centric latents instead of position-sorted autoregressive object generation (ablation) | OCVT · Bouncing Balls Mod3 next-step prediction | color change accuracy: 54.49 -> 82.70 | Helped |
| Use object-centric latents instead of a convolutional feature grid (ablation) | OCVT · Bouncing Balls Mod1234 next-step prediction | color change accuracy: 67.29 -> 78.43 | Helped |
| Add an L1 auxiliary loss for the snitch location (ablation) | OCVT · CATER snitch localization | Top 5 accuracy: 94.4 -> 95.3 | Helped |
| Add an L1 auxiliary loss for the snitch location (ablation) | OCVT · CATER snitch localization | L1 distance: 0.45 -> 0.39 | Helped |
| Add an L1 auxiliary loss for the snitch location (ablation) | OCVT · CATER snitch localization | Top 1 accuracy: 76.0 -> 75.9 | No clear effect |
| Train the full model end-to-end from scratch instead of freezing the pretrained encoder and decoder (ablation) | OCVT · Bouncing Balls Mod1 next-step prediction | color change accuracy: 89.61 -> 82.21\% | Hurt |
| Fine-tune the pretrained encoder and decoder end-to-end instead of freezing them (ablation) | OCVT · Bouncing Balls Mod1 next-step prediction | color change accuracy: 89.61 -> 76.95\% | Hurt |
| Use OCVT rather than the best reported baseline | CATER snitch localization · CATER snitch localization | Top 1 accuracy: 74.8 -> 76.0 | Helped |
Closest papers by meaning
- Snap Video: Scaled Spatiotemporal Transformers for Text-to-Video Synthesis (CVPR 2024 · similarity 0.86)
- Understanding Video Transformers via Universal Concept Discovery (CVPR 2024 · similarity 0.86)
- OmniTokenizer: A Joint Image-Video Tokenizer for Visual Generation (NeurIPS 2024 · similarity 0.85)
- OmniViD: A Generative Framework for Universal Video Understanding (CVPR 2024 · similarity 0.84)
- Learnings from Scaling Visual Tokenizers for Reconstruction and Generation (ICML 2025 · similarity 0.84)
- Don't Look Twice: Faster Video Transformers with Run-Length Tokenization (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.