Research map / Video / Video generation
EDEN: Enhanced Diffusion for High-quality Large-motion Video Frame Interpolation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes EDEN, a DiT-based latent diffusion model for video frame interpolation with a transformer tokenizer, temporal attention, and a start-end frame difference embedding to handle large motion.
Problem
Diffusion-based video frame interpolation still produces blurry, temporally inconsistent frames under large or nonlinear motion, and the diffusion process contributes little to output quality.
Approach
Replaces the 2D-conv VAE with a 1D transformer tokenizer that uses pyramid multi-scale token fusion and temporal attention over start/end frames, pairs it with a DiT that injects start-end context via temporal attention and a normalized start-end cosine-similarity difference embedding in adaLN.
Key results
- On DAVIS, EDEN reaches LPIPS 0.0874 and FloLPIPS 0.1201, versus 0.0963 / 0.1313 for the best prior method LBBDM.
- On DAIN-HD544p, EDEN reaches LPIPS 0.1321 vs 0.1426 for VFIMamba, roughly 8% lower LPIPS.
- EDEN runs at 0.130 s per interpolated frame on DAVIS on a V100, far faster than LDMVFI (7.001 s) and LBBDM (1.689 s); only 2 denoising steps are needed.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add temporal attention in both encoder and decoder blocks (ablation) | EDEN tokenizer · VIMEO | LPIPS: 0.1731 -> 0.0150 | Helped |
| Add temporal attention in both encoder and decoder blocks (ablation) | EDEN tokenizer · VIMEO | FloLPIPS: 0.0473 -> 0.0103 | Helped |
| Replace temporal attention with cross-attention (ablation) | EDEN · DAVIS | LPIPS: 0.0547 vs 0.0548 | No clear effect |
| Replace temporal attention with cross-attention (ablation) | EDEN · DAIN-HD544p | LPIPS: 0.0718 vs 0.0515 | Hurt |
| Increase tokenizer latent dimension from 16 to 24 (ablation) | EDEN tokenizer and diffusion model · DAIN-HD544p | LPIPS: 0.0576 -> 0.0515; 0.1538 -> 0.1641 | Mixed |
| Add Pyramid Feature Fusion Module (ablation) | EDEN tokenizer · DAVIS | LPIPS: 0.0564 -> 0.0511 | Helped |
| Add Pyramid Feature Fusion Module (ablation) | EDEN tokenizer · DAIN-HD544p | LPIPS: 0.0596 -> 0.0428 | Helped |
| Add start-end frame difference embedding (ablation) | EDEN diffusion transformer · DAVIS | LPIPS: 0.0976 -> 0.0874 | Helped |
| Add start-end frame difference embedding (ablation) | EDEN diffusion transformer · DAIN-HD544p | LPIPS: 0.1425 -> 0.1321 | Helped |
Surprising findings
- Decoding random noise instead of the denoised latent yields frames of nearly the same perceptual quality for existing diffusion VFI methods, suggesting the diffusion step contributes little.
Limitations
- Still blurs fast changes in fine details such as text.
- Reconstruction metrics (PSNR/SSIM) not reported, only perceptual metrics.
Experimental setup
Datasets: LAVIB VIMEO DAVIS DAIN-HD544p SNU-FILM
Benchmarks: DAVIS DAIN-HD544p SNU-FILM VIMEO
Metrics: LPIPS FloLPIPS running time
Baselines: VFIFormer ST-MFNet AMT SGM-VFI VFIMamba LDMVFI LBBDM
Builds on: TiTok DiT rectified flow LDMVFI LBBDM
Closest papers by meaning
- Generative Inbetweening: Adapting Image-to-Video Models for Keyframe Interpolation (ICLR 2025 · similarity 0.91)
- Hierarchical Flow Diffusion for Efficient Frame Interpolation (CVPR 2025 · similarity 0.90)
- ViBiDSampler: Enhancing Video Interpolation Using Bidirectional Diffusion Sampler (ICLR 2025 · similarity 0.90)
- Video Interpolation with Diffusion Models (CVPR 2024 · similarity 0.90)
- Efficient Video Diffusion Models via Content-Frame Motion-Latent Decomposition (ICLR 2024 · similarity 0.90)
- Video Motion Transfer with Diffusion Transformers (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.