Research map / Video / Video generation
TPC: Test-time Procrustes Calibration for Diffusion-based Human Image Animation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Test-time Procrustes Calibration (TPC), a plug-and-play method that warps the reference image via Procrustes analysis on keypoints to match each target pose, improving diffusion-based human image animation under compositional (scale/rotation) misalignment without retraining.
Problem
Diffusion-based human image animation degrades in fidelity and temporal consistency when the scale and rotation of the human in the reference image do not match the target pose.
Approach
Calibrate the reference image per target pose with a keypoint-based Procrustes warp (choosing the keypoint subset that maximizes silhouette IoU with the target), then feed the calibrated latents as extra cross-attention conditioning, with iterative propagation of randomly selected calibrated features across frame groups to keep temporal consistency.</δcore_idea>
Key results
- TPC improves MagicAnimate on TED-talks original test set from PSNR 29.17 to 29.52 and FVD from 136 to 128 (with the misaligned split 27.47 to 29.41 PSNR).
- On the compositional-misalignment split of TikTok, DreamPose PSNR rises from 25.17 to 28.01 and FID from 87.2 to 49.3 with TPC.
- TPC keeps PSNR near-optimal down to scale IoU 0.4 versus 0.6 for baselines, per robustness analysis.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add Test-time Procrustes Calibration to MagicAnimate | MagicAnimate · TikTok test (original / compositional misalignment) | PSNR: 29.11/27.82 -> 29.43/29.21 | Helped |
| Add Test-time Procrustes Calibration to MagicAnimate | MagicAnimate · TikTok test (original / compositional misalignment) | FID: 31.8/49.2 -> 29.2/30.4 | Helped |
| Add Test-time Procrustes Calibration to MagicAnimate | MagicAnimate · TED-talks test (original / compositional misalignment) | PSNR: 29.17/27.47 -> 29.52/29.41 | Helped |
| Add Test-time Procrustes Calibration to MagicAnimate | MagicAnimate · TED-talks test (original / compositional misalignment) | FID: 25.7/47.1 -> 24.3/25.6 | Helped |
| Add Test-time Procrustes Calibration to diffusion-based animation baselines | DreamPose, DisCo, AnimateAnyone, and MagicAnimate · TikTok and TED-talks tests | Image and video quality | Helped |
| Maintain calibration for rotated or scaled reference images | TPC-integrated animation baselines · Scale and rotation robustness analysis | PSNR | Helped |
Limitations
- Quality depends on target pose estimation; flicker or low fidelity when target pose frames are poorly specified.
- Large body-shape differences between reference and target still yield awkward transfers.
- Does not handle multiple individuals; conditioning limited to images.
Experimental setup
Datasets: TikTok TED-talks
Benchmarks: TikTok TED-talks
Metrics: L1 PSNR SSIM LPIPS FID FID-VID FVD human preference
Baselines: DreamPose DisCo AnimateAnyone MagicAnimate
Builds on: Procrustes analysis Stable Diffusion DreamPose AnimateAnyone MagicAnimate DisCo
Closest papers by meaning
- MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Model (CVPR 2024 · similarity 0.88)
- HumanVid: Demystifying Training Data for Camera-controllable Human Image Animation (NeurIPS 2024 · similarity 0.87)
- DiffPerformer: Iterative Learning of Consistent Latent Guidance for Diffusion-based Human Video Generation (CVPR 2024 · similarity 0.87)
- Free-viewpoint Human Animation with Pose-correlated Reference Selection (CVPR 2025 · similarity 0.87)
- DisPose: Disentangling Pose Guidance for Controllable Human Image Animation (ICLR 2025 · similarity 0.86)
- AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.