Research map / Detection and segmentation / Tracking
TrajLoc: Trajectory-Attention Localization for Multi-Object Motion Control
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Replaces cross-attention columns of object tokens with trajectory Gaussian heatmaps and encodes trajectory/depth and first-frame appearance as learned object tokens, enabling sparse multi-object motion control in I2V models without dense conditioning tensors.
Problem
Multi-object motion control in image-to-video generation degrades as object count grows and trajectories cross or occlude, because existing methods fuse trajectories into dense video-sized conditioning signals.
Approach
Overwrite each object token's cross-attention column (or, for joint self-attention, an approximation thereof) with a Gaussian heatmap centered on its trajectory at each frame, and carry trajectory+depth via a learned trajectory token and first-frame appearance via a learned appearance token in the text conditioning space.
Key results
- On Wan 14B, Ours achieves PSNR 29.24 on MoVi-Extended vs 20.33 for ATI, and EPE 0.57 vs 2.01.
- Average gain of +4.3 dB PSNR and 51% average EPE reduction versus strongest same-backbone baseline, per the paper.
- Ablation on WaN: removing attention localization drops MoVi PSNR from 29.24 to 22.58 and raises EPE from 0.57 to 1.87.
Surprising findings
- Trained only on synthetic data yet leads baselines trained on real-world video on MOT17 and DAVIS.
- Lowering classifier-free guidance scale from 5.0 to 3.0 reduces first-frame rendering artifacts on the WaN backbone.
Limitations
- Evaluation limited to static-camera scenes at 720x480 resolution.
- Training data predominantly synthetic, causing fallback to synthetic appearance priors (e.g., GTA-V pedestrian sprites) on OOD objects.
- Untracked objects move implausibly; the method has no constraint on them.
Experimental setup
Datasets: MOTSynth MoVi-Extended Pool Football MOT17 DAVIS 2017
Benchmarks: MoVi-Extended Pool Football MOTSynth MOT17 DAVIS 2017
Metrics: PSNR LPIPS FVD end-point error
Baselines: Wan-Move ATI Tora MagicMotion
Builds on: Wan 2.1 CogVideoX Prompt-to-Prompt GLIGEN LoRA
Closest papers by meaning
- Trajectory attention for fine-grained video motion control (ICLR 2025 · similarity 0.88)
- MotionPro: A Precise Motion Controller for Image-to-Video Generation (CVPR 2025 · similarity 0.88)
- Through-The-Mask: Mask-based Motion Trajectories for Image-to-Video Generation (CVPR 2025 · similarity 0.87)
- Wan-Move: Motion-controllable Video Generation via Latent Trajectory Guidance (NeurIPS 2025 · similarity 0.87)
- PoseTraj: Pose-Aware Trajectory Control in Video Diffusion (CVPR 2025 · similarity 0.85)
- Segment Any Motion in Videos (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.