Research map / Image generation / Fast diffusion sampling
What kills $v$-prediction? A Patch-wise PCA Perspective on Pixel-Space Flow Matching
Tianze Luo, Haofeng Huang, Yuan Yao
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Analyzes why v-prediction fails in pixel-space flow matching through a patch-wise PCA lens and proposes a remedy for the resulting training instability.
Problem
Pixel-space flow matching with the standard v-prediction objective often suffers training instabilities, while latent diffusion requires two-stage training and is limited by autoencoder reconstruction.
Approach
Attributes v-prediction failure to a patch-wise PCA structure of pixel data rather than only to the off-manifold nature of noised targets, offering a perspective that explains the instability in pixel space.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| restrict data and noise to low-dimensional principal subspace (PCA 128) (ablation) | v-prediction JiT-B · ImageNet 256 pixel space | FID-50K: 149.07 -> 33.99 | Improved |
| increase PCA dimension from 128 to 384 (ablation) | v-prediction JiT-B · ImageNet 256 | FID-50K: 33.99 -> 47.41 | Hurt |
| increase PCA dimension to full 768 (ablation) | x-prediction JiT-B · ImageNet 256 | FID-50K: 35.01 -> 40.39 | No clear effect |
| increase backbone width to 1024 (JiT-L) (ablation) | vanilla v-prediction · ImageNet 256 | FID-50K: 149.07 -> 33.43 | Improved |
| use patch size 8 in backbone (ablation) | vanilla v-prediction B · ImageNet 256 | FID-50K: 149.07 -> 28.70 | Improved |
| give decoder direct raw-input access (ablation) | width-384 decoder · ImageNet 256 | FID-50K: 186.16 -> 31.85 | Improved |
| increase decoder width from 128 to 256 (ablation) | raw-conditioned decoder · ImageNet 256 | FID-50K: 220.78 -> 34.68 | Improved |
| add output-side shortcut with anchored fractional gains (ablation) | RiT-B decoder · ImageNet 256 | FID-50K: 31.85 -> 27.97 | Improved |
| use separate denominators for raw and semantic branches (ablation) | RiT shortcut · ImageNet 256 | FID-50K: 27.97 -> 28.51 | No clear effect |
| use x-prediction objective with RiT architecture (ablation) | RiT-B · ImageNet 256 | FID-50K: 27.97 -> 28.49 | Hurt |
| reduce backbone input bottleneck width from 768 to 128 (ablation) | RiT-B · ImageNet 256 | FID-50K: 30.45 -> 27.97 | Improved |
| shift time sampling to higher noise and sigma 0.8 (ablation) | RiT-B · ImageNet 256 | FID-50K: 33.69 -> 27.97 | Improved |
Experimental setup
Builds on: flow matching v-prediction
Closest papers by meaning
- Asymmetric Flow Models (NeurIPS 2026 · similarity 0.88)
- Cross Flow: One-Step Generation Across Latent and Pixel Spaces (NeurIPS 2026 · similarity 0.87)
- Balancing Frequencies and Pixels in Flow Matching (NeurIPS 2026 · similarity 0.87)
- Improving Progressive Generation with Decomposable Flow Matching (NeurIPS 2025 · similarity 0.87)
- On the Closed-Form of Flow Matching: Generalization Does Not Arise from Target Stochasticity (NeurIPS 2025 · similarity 0.87)
- Revisiting Diffusion Model Predictions Through Dimensionality (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
