Research map / Video / Video segmentation and tracking
OneVAE: Joint Discrete and Continuous Optimization Helps Discrete Video VAE Train Better
Yupeng Zhou, Zhen Li, Yuming Chen, Ziheng Ouyang, Ruoyi Du, Daquan Zhou, Bin Fu, Yihao Liu, Peng Gao, Ming-Ming Cheng, Qibin Hou
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use FSQ instead of VQ as quantizer (ablation) | continuous VAE latent space · Quantizer on frozen continuous VAE latent | PSNR: 22.82 -> 26.46 | Improved |
| add progressive training from pretrained continuous VAE (vs FSQ from scratch) (ablation) | discrete video VAE · PSNR vs training iterations | convergence speed: 5x convergence speedup | Improved |
| add progressive training from pretrained continuous VAE (vs FSQ from scratch) (ablation) | discrete video VAE · PSNR vs training iterations | PSNR: approximately 1dB | Improved |
| use FSQ instead of SimVQ in progressive training (ablation) | progressive discrete VAE · PSNR vs training iterations | PSNR: SimVQ saturates at a lower PSNR | Improved |
| expand decoder with extra layers (ablation) | 16x16x4 discrete OneVAE · 16x16x4 discrete VAE decoder ablation | PSNR: 24.86 -> 25.05 | Improved |
| expand decoder with extra layers (ablation) | 16x16x4 discrete OneVAE · 16x16x4 discrete VAE decoder ablation | SSIM: 0.7950 -> 0.8003 | Improved |
| add multi-token quantization (two tokens per position) (ablation) | discrete OneVAE 8x8x4 · Panda70M 33x512x512 (discrete) | PSNR: 30.67 -> 31.80 | Improved |
| add multi-token quantization (two tokens per position) (ablation) | discrete OneVAE 8x8x4 · Panda70M 33x512x512 (discrete) | FVD: 108.93 -> 78.35 | Improved |
| add first-frame enhancement (ablation) | 16x16x4 discrete OneVAE · Panda70M 33x512x512 (discrete) | PSNR: 27.40 -> 28.11 | Improved |
| add first-frame enhancement (ablation) | 16x16x4 discrete OneVAE · Panda70M 33x512x512 (discrete) | FVD: 336.48 -> 213.04 | Improved |
| use OneVAE instead of Cosmos Tokenizer (discrete 8x8x4) | discrete video VAE · Panda70M 33x512x512 (discrete) | FVD: 82.86 -> 108.93 | Hurt |
| add joint discrete and continuous dual-path training (ablation) | discrete VAE · latent feature visualization | latent richness | Improved |
Closest papers by meaning
- VQToken: Neural Discrete Token Representation Learning for Extreme Token Reduction in Video Large Language Models (NeurIPS 2025 · similarity 0.88)
- One Token per Highly Selective Frame: Towards Extreme Compression for Long Video Understanding (NeurIPS 2025 · similarity 0.87)
- CV-VAE: A Compatible Video VAE for Latent Generative Video Models (NeurIPS 2024 · similarity 0.87)
- SoftVQ-VAE: Efficient 1-Dimensional Continuous Tokenizer (CVPR 2025 · similarity 0.86)
- DSAQuant: Denoising-Stage-Aligned Quantization-Aware Training for Video Generation (NeurIPS 2026 · similarity 0.86)
- $\text{S}^2$Q-VDiT: Accurate Quantized Video Diffusion Transformer with Salient Data and Sparse Token Distillation (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
