Research map / Video / Video generation
Bidirectional Temporal Diffusion Model for Temporally Consistent Human Animation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a bidirectional temporal diffusion model (BTDM) with a BTU-Net that cross-conditions denoising intermediates of consecutive frames in forward and backward directions to generate temporally coherent human animation from an image, a video, or noise.
Problem
Unidirectional autoregressive generation of human animation suffers from texture drifting and appearance distortion over time due to motion-appearance ambiguity.
Approach
Model human appearance bidirectionally with a denoising diffusion model whose forward and backward per-frame denoising steps are cross-conditioned on each other's noisy intermediates via a shared-weight U-Net with appearance and spatiotemporal bidirectional attention blocks, plus bidirectional recursive sampling at inference.
Key results
- On graphics simulation single-image animation, Ours achieves SSIM 0.958, LPIPS 0.036, tLPIPS 0.003, tOF 8.93, FID 11.14, beating TPS and MRAA on all metrics.
- Bidirectional vs. unidirectional ablation on simulation data: tLPIPS 0.003 vs 0.005 and LPIPS 0.036 vs 0.052.
- On person-specific animation, average LPIPS 4.39 and tLPIPS 3.22 (x10^-3) are competitive with MDMT (4.31/3.29), with the best average SSIM of 9.55 vs 9.49.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use bidirectional rather than unidirectional temporal modeling (ablation) | BTDM · Graphics simulation | SSIM: 0.937 -> 0.958 | Helped |
| Use bidirectional rather than unidirectional temporal modeling (ablation) | BTDM · Graphics simulation | LPIPS: 0.052 -> 0.036 | Helped |
| Use bidirectional rather than unidirectional temporal modeling (ablation) | BTDM · Graphics simulation | tLPIPS: 0.005 -> 0.003 | Helped |
| Use BTDM rather than TPS | Single-image animation · Graphics simulation | SSIM: 0.915 -> 0.958 | Helped |
| Use BTDM rather than TPS | Single-image animation · Graphics simulation | LPIPS: 0.077 -> 0.036 | Helped |
| Use BTDM rather than TPS | Single-image animation · Graphics simulation | tLPIPS: 0.005 -> 0.003 | Helped |
| Use BTDM rather than TPS | Single-image animation · Graphics simulation | tOF: 11.92 -> 8.93 | Helped |
| Use BTDM rather than TPS | Single-image animation · Graphics simulation | FID: 48.76 -> 11.14 | Helped |
| Use BTDM rather than MDMT | Person-specific animation · Person-specific animation, Average | LPIPS (×10²): 4.31 -> 4.39 | Hurt |
| Use BTDM rather than MDMT | Person-specific animation · Person-specific animation, Average | tLPIPS (×10³): 3.29 -> 3.22 | Helped |
| Use BTDM rather than MDMT | Person-specific animation · Person-specific animation, Average | SSIM (×10): 9.49 -> 9.55 | Helped |
| Increase the number of fine-tuning images (ablation) | BTDM single-image animation · Graphics simulation and UBC Fashion | Rendering quality | Helped |
Surprising findings
- The paper reports that the forward-then-backward versus backward-then-forward sampling order made no significant difference in outcomes.
Limitations
- Errors in pose estimation (e.g., hand misdetection) cause appearance distortion.
- 2D pose representation is ambiguous, causing weakness with 3D rotations.
- Inference is slow (about 1.4-1.9 s per frame) and single-image animation requires per-image fine-tuning.
Experimental setup
Datasets: UBC Fashion Graphics simulation dataset (custom, rendered with Character Creator, iClone8, Mixamo motions) Five person-specific videos from prior work
Benchmarks: Graphics simulation dataset UBC Fashion Five person-specific videos
Metrics: SSIM LPIPS tLPIPS tOF FID
Baselines: TPSMM MRAA V2V EDN HFMT DIW MDMT
Builds on: DDPM SR3 Denoising diffusion with predicted clean images (Salimans & Ho style v-predictio FiLM DensePose
Closest papers by meaning
- MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Model (CVPR 2024 · similarity 0.89)
- Human Motion Diffusion as a Generative Prior (ICLR 2024 · similarity 0.87)
- TRIP: Temporal Residual Learning with Image Noise Prior for Image-to-Video Diffusion Models (CVPR 2024 · similarity 0.87)
- Single Motion Diffusion (ICLR 2024 · similarity 0.86)
- AR-Diffusion: Asynchronous Video Generation with Auto-Regressive Diffusion (CVPR 2025 · similarity 0.86)
- FlowMo: Variance-Based Flow Guidance for Coherent Motion in Video Generation (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.