Research map / Video / Video generation
ObjectMover: Generative Object Movement with Video Prior
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Repurposes a 5B image-to-video diffusion transformer as a sequence-to-sequence model for single-image object movement, trained on game-engine synthetic pairs plus real video via multi-task learning.
Problem
Moving an object within an image requires re-harmonizing lighting, adjusting perspective, filling occlusions, synchronizing shadows and reflections, and preserving identity; existing two-step remove-then-insert or copy-paste pipelines fail, and paired training data is scarce.
Approach
Model object movement as a sequence-to-sequence problem by feeding input image, object image, and bounding-box instruction map as frames to a pretrained image-to-video diffusion transformer, with a game-engine synthetic dataset and multi-task training that uses real videos via mask-based insertion.
Key results
- On ObjMove-A object movement, ObjectMover reaches PSNR 25.27, DINO-Score 85.07, DreamSim 0.142, outperforming MagicFixup (23.82, 78.49, 0.198).
- In user study on ObjMove-B, 64.17% of participants preferred ObjectMover versus 31.14% for MagicFixup.
- Adding real video data via mask-based insertion improved DINO-Score from 82.09 vs 80.0-ish range in ablation (Model D 24.24 PSNR, 82.09 DINO) over the synthetic-only variant (Model C 23.89, 79.97).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use an image-to-video checkpoint instead of a text-to-image checkpoint (ablation) | ObjectMover · ObjMove-A | DINO-Score: 75.01 -> 79.97 | Helped |
| add object insertion and removal tasks to synthetic-data training (ablation) | ObjectMover · ObjMove-A | DINO-Score: 80.02 -> 79.97 | No clear effect |
| add real-world video data (ablation) | ObjectMover · ObjMove-A | DINO-Score: 79.97 -> 82.09 | Helped |
| use ObjectMover rather than MagicFixup for object movement | Object movement · ObjMove-A | DINO-Score: 78.49 -> 85.07 | Helped |
| use ObjectMover rather than MagicFixup for object movement | Object movement · ObjMove-B | User-Study: 31.14% -> 64.17% | Helped |
| use ObjectMover rather than LaMa for object removal | Object removal · Object removal | DINO-Score: 63.82 -> 83.94 | Helped |
| use ObjectMover rather than Anydoor for object insertion | Object insertion · Object insertion | DINO-Score: 77.49 -> 85.69 | Helped |
Surprising findings
- Users misidentified the real input image as generated 70% of the time in a self-evaluation game.
Limitations
- Training data is synthetic and has a domain gap from natural images
- Video data is not perfectly aligned with the movement task and requires an auxiliary mask-based insertion task
- Evaluation on ObjMove-A relies on reference target images, which cannot fully capture realism of ill-posed edits
Experimental setup
Datasets: Unreal Engine synthetic object movement data (1,348,248 pairs) real-world natural video data ObjMove-A ObjMove-B
Benchmarks: ObjMove-A ObjMove-B
Metrics: PSNR DINO-Score CLIP-Score DreamSim user study
Baselines: MagicFixup Anydoor Paint-by-Example 3DiT Drag-Anything PowerPaint SD-Inpaint LaMa
Builds on: flow matching latent diffusion Sora-style diffusion transformer PowerPaint MagicFixup
Closest papers by meaning
- Wan-Move: Motion-controllable Video Generation via Latent Trajectory Guidance (NeurIPS 2025 · similarity 0.87)
- DreamScene4D: Dynamic Multi-Object Scene Generation from Monocular Videos (NeurIPS 2024 · similarity 0.87)
- VideoJAM: Joint Appearance-Motion Representations for Enhanced Motion Generation in Video Models (ICML 2025 · similarity 0.87)
- VideoHandles: Editing 3D Object Compositions in Videos Using Video Generative Priors (CVPR 2025 · similarity 0.86)
- Through-The-Mask: Mask-based Motion Trajectories for Image-to-Video Generation (CVPR 2025 · similarity 0.86)
- Motion Modes: What Could Happen Next? (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.