Research map / Video / Video generation
LeanVAE: An Ultra-Efficient Reconstruction VAE for Video Diffusion Models
Yu Cheng, Fajie Yuan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use ISTA-Net+ compressed sensing instead of a two-linear-layer autoencoder for the latent channel bottleneck (ablation) | LeanVAE · DAVIS (4 latent channels, 250k steps) | PSNR / LPIPS: 0.39 dB in PSNR and 0.018 in LPIPS | Improved |
| remove LayerNorm on pixel patches (no patch normalization) (ablation) | LeanVAE · DAVIS (4 latent channels, 250k steps) | PSNR / LPIPS: 3.27 dB in PSNR and 0.013 in LPIPS | Improved |
| process low and high frequency separately then fuse (Variant 2) versus joint processing (ablation) | LeanVAE · DAVIS (4 latent channels, 250k steps) | PSNR / SSIM | Improved |
| add Haar wavelet transform on input (Variant 2 vs Variant 3 without wavelet) (ablation) | LeanVAE · DAVIS (4 latent channels, 250k steps) | reconstruction quality | Improved |
| increase latent channels from 4 to 16 in generation (ablation) | Ours-Latte · UCF101 generation | FVD: 164.45 -> 175.33 | Hurt |
| increase latent channels from 4 to 16 in generation (ablation) | Ours-Latte · SkyTimelapse generation | FVD: 49.59 -> 95.15 | Hurt |
| replace SD VAE with LeanVAE (4 chn) in Latte | Latte-XL · UCF101 generation | FVD: 477.97 -> 164.45 | Improved |
| replace SD VAE with LeanVAE (4 chn) in Latte | Latte-XL · SkyTimelapse generation | FVD: 59.82 -> 49.59 | Improved |
| replace SD VAE with LeanVAE in Latte for training throughput | Latte-XL · training speed | samples/sec: 1.60 -> 6.64 | Improved |
| use temporal tiling with chunksize 5 on VidTok versus full inference | VidTok · DAVIS 17x256x256 | rFVD: 358.28 -> 342.44 | Mixed |
| use temporal tiling with chunksize 5 on LeanVAE versus full inference | LeanVAE · DAVIS 17x256x256 | rFVD: 322.56 vs 322.46 | No clear effect |
| use LeanVAE instead of VidTok at 768x768 resolution | LeanVAE · 768x768 17-frame video | inference time: 20.26 seconds -> 0.46 seconds | Improved |
Closest papers by meaning
- WF-VAE: Enhancing Video VAE by Wavelet-Driven Energy Flow for Latent Video Diffusion Model (CVPR 2025 · similarity 0.92)
- VideoVAE+: Large Motion Video Autoencoding with Cross-modal Video VAE (ICCV 2025 · similarity 0.91)
- CV-VAE: A Compatible Video VAE for Latent Generative Video Models (NeurIPS 2024 · similarity 0.90)
- Improved Video VAE for Latent Video Diffusion Model (CVPR 2025 · similarity 0.90)
- Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Models (CVPR 2025 · similarity 0.87)
- From Slow Bidirectional to Fast Autoregressive Video Diffusion Models (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
