AI Paper Atlas

Research map / Video / Video generation

Speech Drives Templates: Co-Speech Gesture Synthesis With Learned Templates

Shenhan Qian, Zhi Tu, Yihao Zhi, Wen Liu, Shenghua Gao

ICCV 2021 · Video › Video generation · approach: Talking face generation · 79 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add clip-wise template vectors learned by back-propagation (ablation)UNet gesture generator without templates · OliverFTD: 1.66 -> 1.26Improved
add clip-wise template vectors learned by back-propagation (ablation)UNet gesture generator without templates · Olivernormalized lip-sync error: 0.17 -> 0.17No clear effect
learn clip-wise templates with a VAE instead of back-propagation (ablation)template-BP clip-wise · OliverFTD: 1.26 -> 0.92Improved
use frame-wise instead of clip-wise templates (ablation)template-BP · Olivernormalized lip-sync error: 0.17 -> 0.21Hurt
use frame-wise instead of clip-wise templates (ablation)template-BP · OliverFTD: 1.26 -> 0.78Improved
replace batch norm with instance norm (ablation)UNet without templates · OliverFTD: 7.01 -> 1.54Improved
replace batch norm with instance norm (ablation)UNet without templates · Olivernormalized lip-sync error: 0.20 -> 0.19Improved
use transposed instance norm instead of plain instance norm (ablation)UNet without templates · OliverFTD: 1.54 -> 1.53No clear effect
add hierarchical body representation (ablation)UNet without templates (IN*) · Olivernormalized lip-sync error: 0.19 -> 0.17Improved
add hierarchical body representation (ablation)UNet without templates (IN*) · OliverFTD: 1.53 -> 1.66Hurt
use template-VAE instead of MoGlowMoGlow · OliverFTD: 2.28 -> 0.92Improved
use template-VAE instead of Speech2GestureSpeech2Gesture · OliverFTD: 8.30 -> 0.92Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy