Research map / Video / Video generation
Ready-to-React: Online Reaction Policy for Two-Character Interaction Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Ready-to-React, an online reaction policy that couples an autoregressive transformer with a diffusion head over VQ-VAE motion latents, so each boxing character reacts to its opponent's past motion in a streaming fashion.
Problem
Generating two-character interactions online, where each character must react in real time to the counterpart's past motion rather than to complete sequences or jointly generated conditioned sequences, while avoiding error accumulation over long horizons.
Approach
Each character runs the same reaction policy: a transformer history encoder feeds a single-layer MLP diffusion head that predicts the next VQ-VAE motion latent, and a transformer online decoder turns that latent plus recent frames and root info into the next poses, enabling streaming, decision-independent two-character generation.
Key results
- Reactive generation per-clip FID 9.599 vs 13.606 for Duolando-offline and 30.159 for T2MGPT-online; two-character per-frame FID 1.394 vs 2.498 for InterFormer.
- Long-term 1800-frame generation per-clip FID 36.755 vs 87.697 for InterFormer and 273.086 for T2MGPT-online.
- Sparse-signal control per-clip FID 4.086 vs 15.169 for CAMDM, with lower position error (2.72 vs 14.52).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace VQ-VAE motion encoder with VAE (ablation) | Ready-to-React · DuoBox reactive and two-character | FID per-frame: 0.535 -> 0.525; 1.394 -> 1.446 | No clear effect |
| add a motion encoder to predict motion latents instead of raw poses (ablation) | Ready-to-React · DuoBox reactive | FID per-frame: 0.693 -> 0.535 | Helped |
| add a motion encoder to predict motion latents instead of raw poses (ablation) | Ready-to-React · DuoBox two-character | FID per-frame: 7.203 -> 1.394 | Helped |
| replace GPT predictor with diffusion predictor (ablation) | Ready-to-React · DuoBox reactive | FID per-frame: 0.892 -> 0.535 | Helped |
| replace GPT predictor with diffusion predictor (ablation) | Ready-to-React · DuoBox two-character | FID per-frame: 2.418 -> 1.394 | Helped |
| add the online motion decoder instead of decoding with the VQ-VAE decoder (ablation) | Ready-to-React · DuoBox reactive | FID per-frame: 5.215 -> 0.535 | Helped |
| add the online motion decoder instead of decoding with the VQ-VAE decoder (ablation) | Ready-to-React · DuoBox two-character | FID per-frame: 11.279 -> 1.394 | Helped |
| add root sequence R to the decoder input (ablation) | Ready-to-React · DuoBox two-character | FID per-frame: 3.370 -> 1.394 | Helped |
| add root sequence R to the decoder input (ablation) | Ready-to-React · DuoBox reactive | root orientation: 43.3% -> 34.7% | Mixed |
| condition the reaction policy on sparse head and hand signals | Ready-to-React · DuoBox sparse-control reactive | FID per-frame: 0.697 -> 0.249 | Helped |
Surprising findings
- Using a VAE instead of VQ-VAE as motion encoder gives similar results, and removing root R in decoder gives a slightly better per-frame FID (0.496 vs 0.535) in reactive generation.
Limitations
- Limited to two-person interactions; multi-person extension not addressed.
- Does not model interactions with environment or objects.
- Evaluated only on a single boxing dataset of three performers.
Experimental setup
Datasets: DuoBox
Benchmarks: DuoBox
Metrics: FID (per-frame, per-transition, per-clip) jitter root orient (RO) foot sliding (FS) position error rotation error
Baselines: InterFormer CVAE-AR CAMDM T2MGPT-online Duolando-offline
Builds on: VQ-VAE DDIM T2M-GPT Duolando CAMDM
Closest papers by meaning
- InterControl: Zero-shot Human Interaction Generation by Controlling Every Joint (NeurIPS 2024 · similarity 0.88)
- ReGenNet: Towards Human Action-Reaction Synthesis (CVPR 2024 · similarity 0.88)
- AvatarGO: Zero-shot 4D Human-Object Interaction Generation and Animation (ICLR 2025 · similarity 0.87)
- InterDreamer: Zero-Shot Text to 3D Dynamic Human-Object Interaction (NeurIPS 2024 · similarity 0.86)
- Digital Life Project: Autonomous 3D Characters with Social Intelligence (CVPR 2024 · similarity 0.86)
- Sitcom-Crafter: A Plot-Driven Human Motion Generation System in 3D Scenes (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.