Research map / Video / Video generation
Optimizing Diffusion Noise Can Serve As Universal Motion Priors
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Diffusion Noise Optimization (DNO), which optimizes the initial diffusion noise of a frozen pretrained text-to-motion model through the full ODE denoising chain so that any differentiable motion-space loss can drive editing, refinement and completion without retraining.
Problem
Existing motion diffusion models are tied to task-specific conditioning or require retraining per task; there is no general way to reuse a pretrained motion diffusion model as a motion prior for diverse motion editing, control, denoising and completion tasks.
Approach
Treat the pretrained diffusion model plus DDIM ODE solver as a differentiable generator and optimize its latent noise x_T by backpropagating a task criterion computed on the final motion, with normalized gradients, a latent decorrelation loss, and inversion-based initialization for editing.
Key results
- In motion editing on 96 edits, DNO-MDM reaches 0.92-0.95 content preservation and 0.00 m objective error versus GMD's 0.59-0.79 and 0.22 m on jumping/crawling/raised-hand actions.
- For noisy motion refinement (5 cm noise, all joints), DNO-GMD achieves MPJPE 7.0 cm and FID 0.10, versus GMD at 25.7 cm MPJPE and HuMoR at 7.2 cm.
- Ablation shows normalized gradients reduce MPJPE from 30.2 cm to 8.7 cm on the surrogate refinement task.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use DNO-MDM Edit rather than GMD | motion editing · jumping, doing a long jump, walking with raised hand, and crawling | Content preservation ratio: 0.64 -> 0.95; 0.59 -> 0.92; 0.65 -> 0.92; 0.79 -> 0.94 | Helped |
| use DNO-MDM Edit rather than GMD | motion editing · jumping, doing a long jump, walking with raised hand, and crawling | Objective error (m): 0.22 -> 0.00; 0.22 -> 0.00; 0.16 -> 0.00; 0.15 -> 0.00 | Helped |
| use DNO-GMD rather than GMD | noisy motion refinement · HumanML3D, all joints | MPJPE observed (cm): 25.7 -> 7.0 | Helped |
| use DNO-GMD rather than GMD | noisy motion refinement · HumanML3D, all joints | FID: 6.91 -> 0.10 | Helped |
| remove gradient normalization (ablation) | DNO · noisy motion refinement, std. 1 cm, HumanML3D | MPJPE all (cm): 8.7 -> 30.2 | Hurt |
| remove latent decorrelation loss (ablation) | DNO · noisy motion refinement, std. 1 cm, HumanML3D | Foot skating ratio: 0.07 -> 0.10 | Hurt |
| remove latent decorrelation loss (ablation) | DNO · noisy motion refinement, std. 1 cm, HumanML3D | Jitter: 0.33 -> 0.40 | Hurt |
| add random perturbation γ = 2 × 10−4 (ablation) | DNO · noisy motion refinement, std. 1 cm, HumanML3D | FID: 0.66 -> 0.75 | Hurt |
| add random perturbation γ = 5 × 10−4 (ablation) | DNO · noisy motion refinement, std. 1 cm, HumanML3D | FID: 0.66 -> 0.73 | Hurt |
| add random perturbation γ = 10−3 (ablation) | DNO · noisy motion refinement, std. 1 cm, HumanML3D | FID: 0.66 -> 0.91 | Hurt |
| optimize for 300 rather than 500 steps (ablation) | DNO · noisy motion refinement, std. 1 cm, HumanML3D | MPJPE all (cm): 8.7 -> 12.1 | Hurt |
| optimize for 700 rather than 500 steps (ablation) | DNO · noisy motion refinement, std. 1 cm, HumanML3D | MPJPE all (cm): 8.7 -> 7.2 | Helped |
Surprising findings
- Optimizing with the gradient-normalized update and a decorrelation loss matters greatly; without normalized gradients MPJPE rises from 8.7 to 30.2 cm.
- Random perturbation during optimization did not help and larger perturbation worsened results, so the final method uses none.
Limitations
- Quality is bounded by the underlying diffusion model's performance.
- Optimization is slow, limiting interactive editing.
- Works poorly when the observation has poor body coverage, and sometimes cannot project to realistic motion.
Experimental setup
Datasets: HumanML3D
Benchmarks: HumanML3D test subset
Metrics: Jitter foot skating ratio content preservation ratio objective error MPJPE FID
Baselines: GMD HuMoR SDEdit MLD AGRoL AvatarPoser
Builds on: MDM DDIM EDICT DOODL StyleGAN2 latent optimization
Closest papers by meaning
- Human Motion Diffusion as a Generative Prior (ICLR 2024 · similarity 0.89)
- MotionMix: Weakly-Supervised Diffusion for Controllable Motion Generation (AAAI 2024 · similarity 0.88)
- Motion-Zero: A Zero-Shot Trajectory Control Framework of Moving Object for Diffusion-Based Video Generation (AAAI 2025 · similarity 0.88)
- CoDA: Coordinated Diffusion Noise Optimization for Whole-Body Manipulation of Articulated Objects (NeurIPS 2025 · similarity 0.88)
- Go-with-the-Flow: Motion-Controllable Video Diffusion Models Using Real-Time Warped Noise (CVPR 2025 · similarity 0.88)
- Towards Detailed Text-to-Motion Synthesis via Basic-to-Advanced Hierarchical Diffusion Model (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.