Research map / Video / Video generation
PACER+: On-Demand Pedestrian Animation Controller in Driving Scenarios
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PACER+, a physics-based pedestrian animation controller that jointly tracks partial-body reference motion (masked per joint and time) and follows 2D trajectories on terrain with a single PPO/AMP policy.
Problem
Pedestrian animation in driving simulators is either trajectory-following only (basic locomotion) or limited to reproducing reference videos per-sequence, lacking diverse, controllable, on-demand human motion content.
Approach
Combines motion imitation and trajectory following in one goal-conditioned policy using a per-joint spatial-temporal mask that specifies which body parts and time steps must track a reference motion, randomly sampled during training, enabling zero-shot control with motions from generative models, mocap, or videos.
Key results
- Versus PACER, FID drops from 7.97 to 6.74 and diversity rises from 1.29 to 1.67 on synthesized trajectories.
- Ablation: adding motion tracking with the spatial-temporal mask lowers Empjpe from 215.55 to 77.87 mm.
- On real-world Waymo scenes, Empjpe improves from 89.42 (Wang et al.) to 77.67 mm, with lower foot sliding (7.68 vs 7.87).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace PACER with PACER+ | Pedestrian animation on synthetic trajectories at normal speed · Synthetic trajectories, normal speed | FID: 7.97 -> 6.74 | Helped |
| replace PACER with PACER+ | Pedestrian animation on synthetic trajectories at normal speed · Synthetic trajectories, normal speed | Diversity: 1.29 -> 1.67 | Helped |
| replace PACER with PACER+ | Pedestrian animation on synthetic trajectories at low speed · Synthetic trajectories, low speed | l-FID: 8.84 -> 7.62 | Helped |
| replace PACER with PACER+ | Pedestrian animation on synthetic trajectories at low speed · Synthetic trajectories, low speed | l-Diversity: 1.24 -> 1.36 | Helped |
| use PACER+ instead of Wang | Whole-body motion tracking · Motion tracking, whole body | Empjpe: 80.29 -> 72.10 | Helped |
| use PACER+ instead of Wang | Whole-body motion tracking · Motion tracking, whole body | Egmpjpe: 137.48 -> 123.88 | Helped |
| use PACER+ rather than Wang | Real-world pedestrian animation recreation · Real-world scenarios | Empjpe: 89.42 -> 77.67 | Helped |
| use PACER+ rather than Wang | Real-world pedestrian animation recreation · Real-world scenarios | Egmpjpe: 137.84 -> 127.84 | Helped |
| use PACER+ rather than Wang | Real-world pedestrian animation recreation · Real-world scenarios | FS: 7.87 -> 7.68 | Helped |
| use PACER+ rather than Wang | Real-world pedestrian animation recreation · Real-world scenarios | FL: 14.21 -> 12.12 | Helped |
| add motion tracking (ablation) | PACER+ for upper-body tracking and trajectory following · Upper-body tracking and trajectory following | FID: 7.97 -> 7.07 | Helped |
| add motion tracking (ablation) | PACER+ for upper-body tracking and trajectory following · Upper-body tracking and trajectory following | Empjpe: 215.55 -> 79.57 | Helped |
Limitations
- Relies on pre-trained motion generation models for motion content and user-provided trajectories.
- Does not explicitly model semantic relationships between pedestrians and the environment.
- Evaluated only on a small set of baselines and mostly on synthetic terrains; motion content quality depends on external generators.
Experimental setup
Datasets: AMASS Waymo Open Dataset MatrixCity
Benchmarks: AMASS motion tracking Waymo real-world video recreation
Metrics: FID Diversity Empjpe Egmpjpe foot sliding foot penetration velocity error acceleration error
Baselines: PACER Wang et al. (2023) video-based simulation motion capture NIKI
Builds on: PACER PHC AMP PPO NIKI
Closest papers by meaning
- Learning to Generate Diverse Pedestrian Movements from Web Videos with Noisy Labels (ICLR 2025 · similarity 0.84)
- UniMotion: A Unified Motion Framework for Simulation, Prediction and Planning (NeurIPS 2025 · similarity 0.84)
- Continuous Locomotive Crowd Behavior Generation (CVPR 2025 · similarity 0.84)
- DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generation (AAAI 2025 · similarity 0.83)
- MotionPro: A Precise Motion Controller for Image-to-Video Generation (CVPR 2025 · similarity 0.83)
- ReSim: Reliable World Simulation for Autonomous Driving (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.