Research map / Video / Video generation
REGEN: Learning Compact Video Embedding with (Re-)Generative Decoder
Yitian Zhang, Long Mai, Aniruddha Mahapatra, David Bourgin, Yicong Hong, Jonah Casebeer, Feng Liu, Yun Fu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add content-aware positional conditioning (vs in-context conditioning) (ablation) | REGEN · DAVIS 2019 384x640 unseen resolution | rFVD: 441.98 -> 57.01 | Improved |
| add content-aware positional conditioning (vs in-context conditioning) (ablation) | REGEN · DAVIS 2019 192x320 training resolution | rFVD: 135.89 -> 128.80 | Improved |
| add content-aware positional conditioning (vs in-context conditioning) (ablation) | REGEN · DAVIS 2019 384x640 unseen resolution | PSNR: 23.39 -> 29.41 | Improved |
| replace MAGVIT-v2 decoder with diffusion decoder at 32x temporal compression | MAGVIT-v2 · DAVIS 2019 256x256 | rFVD: 1080.15 -> 488.89 | Improved |
| replace MAGVIT-v2 decoder with diffusion decoder at 8x temporal compression | MAGVIT-v2 · MCL-JCV 256x256 | PSNR: 26.61 -> 28.82 | Improved |
| replace MAGVIT-v2 with REGEN at base 4x compression | MAGVIT-v2 · MCL-JCV 512x512 | rFVD: 28.63 -> 22.40 | Improved |
| widen MAGVIT-v2 decoder at 16x compression (ablation) | MAGVIT-v2 · DAVIS 2019 256x256 | rFVD: 441.24 -> 486.48 | Hurt |
| widen MAGVIT-v2 decoder at 32x compression (ablation) | MAGVIT-v2 · DAVIS 2019 256x256 | rFVD: 1080.15 -> 1020.80 | Improved |
| increase diffusion sampling steps (ablation) | REGEN 4x · DAVIS 2019 and MCL-JCV | PSNR / rFVD: slightly worse PSNR, better rFVD | Mixed |
| reduce sampling to one step (ablation) | REGEN 4x · reconstruction examples (4x model) | visual quality | No clear effect |
| add latent extension (SDEdit-style) across chunks (ablation) | REGEN · chunk boundary smoothness | x-t slice jumping | Improved |
| use REGEN decoder latency vs MAGVIT-v2 at 8x8x8 | MAGVIT-v2 · decoder latency 256x256 | latency (ms): 317 -> 159 | Improved |
Closest papers by meaning
- Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Models (CVPR 2025 · similarity 0.89)
- Efficient Video Diffusion Models via Content-Frame Motion-Latent Decomposition (ICLR 2024 · similarity 0.89)
- Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think (ICLR 2025 · similarity 0.88)
- Boosting Generative Image Modeling via Joint Image-Feature Synthesis (NeurIPS 2025 · similarity 0.88)
- Masked Autoencoders Are Effective Tokenizers for Diffusion Models (ICML 2025 · similarity 0.88)
- CV-VAE: A Compatible Video VAE for Latent Generative Video Models (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
