Research map / Video / Video generation
Beyond Distribution Matching: Self-Supervised Representation Forcing for Few-Step Video Generation
Junyi Chen, Haiyu Zhang, ZHOUJIE FU, Tengfei Wang, Chunchao Guo
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a self-supervised representation forcing objective added to Distribution Matching Distillation to improve few-step video diffusion generation.
Problem
Distribution Matching Distillation (DMD) for few-step video diffusion models optimizes only in output space and gives no signal shaping the student's internal representations.
Approach
Adds a self-supervised representation-level training signal to DMD so that the student's internal features are explicitly shaped, rather than relying solely on output-space distribution matching; applicable to both bidirectional and causal student architectures.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add self-supervised representation loss (EMA teacher, heterogeneous noise) to DMD (ablation) | DMD on HunyuanVideo-1.5 · 3D re-rendering probe, 120 static-scene prompts | PSNR: 20.64 -> 21.24 | Improved |
| add representation loss to DMD (ablation) | DMD on HunyuanVideo-1.5 · VBench Temporal Flickering | Temporal Flickering: 96.39 -> 96.86 | Improved |
| add representation loss to DMD (ablation) | DMD on HunyuanVideo-1.5 · in-scene text rendering (qualitative) | text legibility | Improved |
| add representation loss to causal Self-Forcing-style distillation | Causal-Forcing student, Wan2.1-1.3B · VBench Total | VBench Total score: 84.04 -> 84.15 | Improved |
| add representation loss to bidirectional DMD | Wan2.1-1.3B bidirectional student · VisionReward (100 prompts) | VisionReward score: 5.310 -> 7.060 (vs CausVid) | Improved |
| set masking ratio max to 0.4 (bidirectional) or 0.2 (causal) instead of other values in 0-1.0 (ablation) | SF2 masking ratio rho_max · 400-prompt motion-rich validation set | VBench-style score | Improved |
| use space-only mask sampling instead of agnostic per-token masking (ablation) | SF2 causal student · 400-prompt motion-rich validation set | Dynamic Degree vs Background Consistency | Mixed |
| add SF2 to WorldPlay distillation pipeline | WorldPlay · interactive camera/action following | translation, rotation and action accuracy | Improved |
| add SF2 branch to DMD training | DMD training cycle · per-cycle wall-clock | % of cycle: SF2 branch 6.2 of 100.0 percent of cycle | No clear effect |
| do not reset projector learning rate multiplier after warmup (ablation) | SF2 projector warmup · training stability | training stability | Hurt |
| do not stop gradient on teacher path (ablation) | SF2 EMA teacher · training stability | EMA update stability | Hurt |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- Causal-Forcing — Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generation (ICML 2026) · better on 4 of 4 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Surprising findings
- DMD has a structural blind spot: its objective lives entirely in output space and provides no signal pushing the student's internal representations.
Experimental setup
Baselines: Distribution Matching Distillation
Builds on: Distribution Matching Distillation
Closest papers by meaning
- CoDMD: Copula-aware Distribution Matching Distillation for Fast Video Generation (NeurIPS 2026 · similarity 0.89)
- From Slow Bidirectional to Fast Autoregressive Video Diffusion Models (CVPR 2025 · similarity 0.88)
- Motion Consistency Model: Accelerating Video Diffusion with Disentangled Motion-Appearance Distillation (NeurIPS 2024 · similarity 0.88)
- Emergent Temporal Correspondences from Video Diffusion Transformers (NeurIPS 2025 · similarity 0.88)
- Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion (NeurIPS 2025 · similarity 0.87)
- Continuous-Time Distribution Matching for Few-Step Diffusion Distillation (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
