Research map / Video / Video generation

GLARE: Generating Listening Heads with Appropriate REactions

NeurIPS 2026 · Video › Video generation · approach: Talking face generation · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Curates a 147-hour listener-reaction dataset with 64,557 event annotations, proposes GLARE (flow-matching listening-head generator with Qwen2-Audio prosody conditioning and a temporal reaction loss), and four reaction-centric metrics (R-F1, R-tIoU, R-ATD, R-FID).

Problem

Listening head generation lacks fine-grained listener reaction annotations and evaluation metrics; existing metrics measure visual realism rather than whether a listener reacts with the right type, timing, and visual quality.

Approach

Combines an event-level reaction-annotated dyadic dataset, a flow-matching transformer listener with prosody conditioning from an audio LLM and a frame-wise reaction supervision loss, and event-based metrics (matching by class and tIoU, asymmetric timing penalty, reaction-region FID).

Key results

• On RealTalk, GLARE reaches R-F1 0.245 (vs DyStream 0.208 DI-Sync; R-F1 0.594 vs 0.535), R-tIoU 0.704, R-ATD 57.388 and FID 35.697 vs DyStream 37.416.
• On Seamless, GLARE improves R-F1 to 0.460 from 0.414 for DyStream and reduces R-FID to 28.857 from 30.774.
• Human evaluation over 165 reaction events gives HAR of 89.9% naturalness, 89.4% contextual appropriateness, and 93.5% timing plausibility.

Surprising findings

• Prosody conditioning alone improves R-F1 across reaction types but degrades temporal metrics (R-tIoU and R-ATD) for most categories.
• The DI-Sync and R-F1 values reported in the RealTalk comparison show GLARE best on most metrics, yet DyStream attains higher Var on RealTalk and ListenFormer higher DI-Sync on Seamless.

Limitations

• Evaluation is against a single ground-truth reference, which may penalize plausible alternative reactions.
• Reaction annotations come from an automatic detector with manual verification, so metrics depend on the detector's accuracy.
• Surprised and laughing reactions show lower contextual appropriateness and are harder to model.

Experimental setup

Datasets: RealTalk Seamless Interaction ViCo

Benchmarks: RealTalk Seamless Interaction

Metrics: PSNR SSIM FID FVD Var LPIPS rPCC DI-Sync R-F1 R-tIoU

Baselines: L2L DIM ViCo ListenFormer DyStream

Builds on: FLOAT LIA Wav2Vec2 DyStream

Closest papers by meaning

• OmniResponse: Online Multimodal Conversational Response Generation in Dyadic Interactions (NeurIPS 2025 · similarity 0.88)
• INFP: Audio-Driven Interactive Head Generation in Dyadic Conversations (CVPR 2025 · similarity 0.88)
• DualTalk: Dual-Speaker Interaction for 3D Talking Head Conversations (CVPR 2025 · similarity 0.87)
• EmotiveTalk: Expressive Talking Head Generation through Audio Information Decoupling and Emotional Video Diffusion (CVPR 2025 · similarity 0.87)
• The Audio-Visual Conversational Graph: From an Egocentric-Exocentric Perspective (CVPR 2024 · similarity 0.85)
• CustomListener: Text-guided Responsive Interaction for User-friendly Listening Head Generation (CVPR 2024 · similarity 0.85)