Research map / Image generation / Text-to-image diffusion
SoPo: Text-to-Motion Generation Using Semi-Online Preference Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SoPo, a DPO-based preference optimization for text-to-motion diffusion models that pairs offline preferred motions with online-generated unpreferred motions, improving MLD and MDM on HumanML3D and KIT-ML.
Problem
Text-to-motion models produce inconsistent, unrealistic motions, and existing DPO-based preference alignment (MoDiPO) suffers from overfitting in offline DPO and biased sampling in online DPO.
Approach
Train on "semi-online" pairs: preferred motions from the offline dataset and unpreferred motions sampled online from the policy (lowest reward among K samples), with a threshold filter on unpreferred quality and a cosine-similarity-based reweighting of preferred motions.
Key results
- On HumanML3D, MLD* + SoPo reaches R-Precision Top-1 0.528 (vs 0.504 baseline) and MM Dist 2.939 (vs 3.052), with FID 0.174 vs 0.450.
- MDM (fast) + SoPo improves MM Dist by 2.91% (3.304 to 3.208 in the table) and R-Precision Top-1 by 5.27% over the fast MDM baseline, while MoDiPO variants yield only about 0.76% MM Dist gain.
- SoPo fine-tuning takes about 100 minutes on a single RTX 4090D and cuts preference data generation time to roughly 1/10 of MoDiPO's.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune MLD with SoPo instead of MoDiPO-T | MLD · HumanML3D test | MM Dist: 3.267$^{\pm .010}$ -> 3.185$^{\pm 0.012}$ | Helped |
| Fine-tune MDM (fast) with SoPo | MDM (fast) · HumanML3D test | R-Precision Top 1: 0.455 -> 0.479 | Helped |
| Fine-tune MLD* with SoPo | MLD* · HumanML3D test | FID: 0.450 -> 0.174 | Helped |
| Fine-tune MoMask with SoPo | MoMask · KIT-ML test | FID: 0.204 -> 0.176 | Helped |
| Use DSoPo objective instead of full SoPo (ablation) | MDM (fast) with SoPo · HumanML3D test | R-Precision Top 1: .479 -> .460 | Hurt |
| Remove value-unpreferred (VU) component (ablation) | MDM (fast) with SoPo · HumanML3D test | R-Precision Top 1: .479 -> .460 | Hurt |
| Use USoPo without offline preferred-motion reweighting (ablation) | MDM (fast) with SoPo · HumanML3D test | R-Precision Top 1: .479 -> .473 | Hurt |
| Set threshold tau to 0.40 instead of 0.45 (ablation) | MDM (fast) with SoPo · HumanML3D test | R-Precision Top 1: .479 -> .475 | Hurt |
| Set threshold tau to 0.50 instead of 0.45 (ablation) | MDM (fast) with SoPo · HumanML3D test | R-Precision Top 1: .479 -> .468 | Hurt |
| Set threshold tau to 0.55 instead of 0.45 (ablation) | MDM (fast) with SoPo · HumanML3D test | R-Precision Top 1: .479 -> .466 | Hurt |
| Set threshold tau to 0.60 instead of 0.45 (ablation) | MDM (fast) with SoPo · HumanML3D test | R-Precision Top 1: .479 -> .461 | Hurt |
| Set candidate sample count K to 2 instead of 4 (ablation) | MDM (fast) with SoPo · HumanML3D test | R-Precision Top 1: .479 -> .480 | No clear effect |
Surprising findings
- Naive combination of online and offline DPO (Com. DPO) gives less improvement than the proposed semi-online SoPo design, and pure online DPO increases motion diversity drop relative to the base model.
- MoDiPO-O reduces MM Dist and R-Precision relative to the base MLD model while only improving FID.
Limitations
- Relies on a motion reward model (TMR) whose quality and generalization are limited, which SoPo inherits.
- Reward model trained on specific datasets; difficult to extend to open-domain diffusion fine-tuning.
- Hyperparameters K and tau are tuned via preliminary experiments; results vary with tau (lower tau improves R-Precision but hurts FID/diversity).
Experimental setup
Datasets: HumanML3D KIT-ML HPDv2
Benchmarks: HumanML3D KIT-ML HPD dataset
Metrics: R-Precision MM Dist Diversity FID Multimodal Dist HPS CLIP score PickScore ImageReward UnifiedReward
Baselines: MoDiPO-T MoDiPO-G MoDiPO-O MLD MDM MotionGPT MotionDiffuse Motion Mamba MoMask Offline DPO Online DPO
Builds on: DPO MoDiPO Plackett-Luce model RLHF
Closest papers by meaning
- DrPO: Drifting Preference Optimization for One-Step Generative Models (NeurIPS 2026 · similarity 0.87)
- VideoDPO: Omni-Preference Alignment for Video Diffusion Generation (CVPR 2025 · similarity 0.87)
- Smoothed Preference Optimization via ReNoise Inversion for Aligning Diffusion Models with Varied Human Preferences (ICML 2025 · similarity 0.87)
- Curriculum Direct Preference Optimization for Diffusion and Consistency Models (CVPR 2025 · similarity 0.86)
- Aesthetic Post-Training Diffusion Models from Generic Preferences with Step-by-step Preference Optimization (CVPR 2025 · similarity 0.86)
- OMG: Towards Open-vocabulary Motion Generation via Mixture of Controllers (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.