Research map / Video / Video generation
ReGenNet: Towards Human Action-Reaction Synthesis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces a multi-setting human action-reaction synthesis benchmark with actor-reactor annotations on NTU120, InterHuman and Chi3D, and ReGenNet, an online diffusion Transformer-decoder model with an explicit distance-based interaction loss.
Problem
Generating plausible, instant human reactions conditioned on another person's action sequence, which existing human motion generation and human-scene/object interaction methods do not model because they ignore the asymmetric actor-reactor roles and online constraints.
Approach
A diffusion model with a Transformer decoder using a directional (causal) attention mask so reactions depend only on past and current actor motion, trained with an explicit relative-distance interaction loss on actor-reactor annotated SMPL-X data.
Key results
- On NTU120-AS (online, unconstrained, test-conditioned), ReGenNet achieves FID 11.00 vs 24.25 for MDM-GRU and 54.54 for MDM, with accuracy 0.749.
- On Chi3D-AS train-conditioned, ReGenNet reaches FID 0.27 vs 4.96 for the best baseline MDM-GRU.
- On InterHuman-AS, ReGenNet obtains FID 2.265 and R Precision 0.407, outperforming InterGen (13.237 FID, 0.374 R Precision).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace concatenation of actor and reactor features with addition (ablation) | ReGenNet · NTU120-AS, test-conditioned | FID: 10.999±0.7425 -> 12.828±0.9335 | Hurt |
| Remove positional embedding (ablation) | ReGenNet · NTU120-AS, test-conditioned | FID: 10.999±0.7425 -> 12.916±1.2802 | Hurt |
| Add explicit interaction loss (ablation) | ReGenNet · NTU120-AS, test-conditioned | FID: 12.146±0.9044 -> 10.999±0.7425 | Helped |
| Use 2 decoder layers instead of 8 (ablation) | ReGenNet · NTU120-AS, test-conditioned | FID: 10.999±0.7425 -> 29.015±3.7751 | Hurt |
| Use 4 decoder layers instead of 8 (ablation) | ReGenNet · NTU120-AS, test-conditioned | FID: 10.999±0.7425 -> 18.148±1.7413 | Hurt |
| Use 12 decoder layers instead of 8 (ablation) | ReGenNet · NTU120-AS, test-conditioned | FID: 10.999±0.7425 -> 13.348±0.7420 | Hurt |
| Use 16 decoder layers instead of 8 (ablation) | ReGenNet · NTU120-AS, test-conditioned | FID: 10.999±0.7425 -> 11.089±1.3902 | No clear effect |
| Use 2 DDIM sampling steps instead of 5 (ablation) | ReGenNet · NTU120-AS, test-conditioned | FID: 10.999±0.7425 -> 12.132±0.8301 | Hurt |
| Use 10 DDIM sampling steps instead of 5 (ablation) | ReGenNet · NTU120-AS, test-conditioned | FID: 10.999±0.7425 -> 11.466±0.8199 | No clear effect |
| Use 100 DDIM sampling steps instead of 5 (ablation) | ReGenNet · NTU120-AS, test-conditioned | FID: 10.999±0.7425 -> 11.082±0.7440 | No clear effect |
| Use 1000 DDIM sampling steps instead of 5 (ablation) | ReGenNet · NTU120-AS, test-conditioned | FID: 10.999±0.7425 -> 11.719±0.8059 | No clear effect |
| Annotate actor-reactor order rather than randomly shuffling roles (ablation) | ReGenNet · NTU120-AS | FID: 12.69±1.22 -> 11.00±0.74 | Helped |
Surprising findings
- Randomly shuffling actor-reactor roles degrades FID from 11.00 to 12.69 and accuracy from 0.749 to 0.656, showing the annotations matter.
- Using 5 DDIM steps gave the best FID with low latency, better than 1000 sampling steps.
Limitations
- Only atomic interaction periods are modeled, not long-duration multi-transition interactions.
- NTU120 SMPL-X motions are noisy, with inter-penetrations and unnatural facial expressions.
- Chi3D-AS test set is small, so results may fluctuate.
Experimental setup
Datasets: NTU120-AS InterHuman-AS Chi3D-AS
Benchmarks: NTU120-AS Chi3D-AS InterHuman-AS
Metrics: FID action recognition accuracy diversity multimodality R Precision MM Dist
Baselines: cVAE AGRoL MDM MDM-GRU T2M RAIG InterGen
Builds on: MDM AGRoL DDIM classifier-free guidance
Closest papers by meaning
- Ready-to-React: Online Reaction Policy for Two-Character Interaction Generation (ICLR 2025 · similarity 0.88)
- InterDreamer: Zero-Shot Text to 3D Dynamic Human-Object Interaction (NeurIPS 2024 · similarity 0.88)
- InterControl: Zero-shot Human Interaction Generation by Controlling Every Joint (NeurIPS 2024 · similarity 0.87)
- TIMotion: Temporal and Interactive Framework for Efficient Human-Human Motion Generation (CVPR 2025 · similarity 0.87)
- HOI-Dyn: Learning Interaction Dynamics for Human-Object Motion Diffusion (NeurIPS 2025 · similarity 0.87)
- InterMask: 3D Human Interaction Generation via Collaborative Masked Modeling (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.