Research map / Video / Video generation
HiTVideo: Hierarchical Tokenizers for Enhancing Text-to-Video Generation with Autoregressive Large Language Models
Ziqin Zhou, Yifan Yang, Yuqing Yang, Tianyu He, Houwen Peng, Kai Qiu, Qi Dai, Lili Qiu, Chong Luo, Lingqiao Liu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use multi-layer hierarchical codebook instead of single-layer codebook at equal token count (ablation) | HiTVideo tokenizer · Pexels reconstruction | PSNR: 26.98 -> 27.53 | Improved |
| use multi-layer codebook instead of single layer (ablation) | HiTVideo tokenizer with Llama-3B · Pexels text-to-video | FVD: 3,020 -> 1,140 | Improved |
| use multi-layer codebook instead of single layer (ablation) | HiTVideo tokenizer with Llama-3B · Pexels text-to-video | CLIPSIM: 0.16 -> 0.29 | Improved |
| use dense single-layer codebook for LLM generation of 64-frame videos (ablation) | Llama-3B · Pexels text-to-video | qualitative coherence | Hurt |
| raise input resolution from 128 to 256 with higher compression for same token count (ablation) | HiTVideo tokenizer · Pexels reconstruction | PSNR convergence | Improved |
| apply masked decoding (repeated, zero, or learned mask tokens) to a single codebook (ablation) | single-layer tokenizer · Pexels reconstruction | PSNR | Hurt |
| apply masked decoding to the hierarchical tokenizer (masking ratio up to 85%) (ablation) | HiTVideo tokenizer · Pexels reconstruction | PSNR | No clear effect |
| use hierarchical tokenizer with 8s 64-frame input | video tokenizer · Pexels | bits per pixel: 0.038 -> 0.012 | Improved |
Closest papers by meaning
- LARP: Tokenizing Videos with a Learned Autoregressive Generative Prior (ICLR 2025 · similarity 0.90)
- A Recipe for Scaling up Text-to-Video Generation with Text-free Videos (CVPR 2024 · similarity 0.90)
- OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generation (ICLR 2025 · similarity 0.89)
- LinGen: Towards High-Resolution Minute-Length Text-to-Video Generation with Linear Computational Complexity (CVPR 2025 · similarity 0.89)
- VideoTetris: Towards Compositional Text-to-Video Generation (NeurIPS 2024 · similarity 0.89)
- CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
