Research map / Image generation / Fast diffusion sampling
ReDi: Rectified Discrete Flow
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Rectified Discrete Flow (ReDi), an iterative coupling-rectification method that provably decreases a conditional total-correlation factorization error, enabling few-step and one-step discrete flow generation on ImageNet and OpenWebText.
Problem
Discrete flow-based models need many sampling steps because the per-dimension factorization approximation error grows with large step sizes, hindering few-step and one-step generation.
Approach
Characterize factorization error via conditional total correlation, show it depends on the coupling between source and target, and iteratively rectify the coupling by retraining a DFM on model-generated (source, target) pairs; each rectification provably does not increase conditional TC. Rectified couplings are then used to train one-step models without teacher-student setups or special objectives
Key results
- On ImageNet 4-step generation, ReDi^1 reaches FID 7.58 and IS 228 versus MaskGIT's 10.90 FID and 184 IS.
- ReDi^3-distill achieves 1-step FID 11.68 and IS 182, versus SDTT's 90.40 and Di4C's 90.32 FID at one step.
- On OpenWebText, ReDi variants reach teacher-level generative perplexity at 16 and 8 steps, giving reported 64x and 128x speedups over the 1024-step teacher.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Rectify the coupling before one-step generation | MaskGIT · ImageNet 1-step | FID: 95.16 -> 11.68 | Helped |
| Rectify the coupling once | MaskGIT · ImageNet 4-step | FID: 10.90 -> 7.58 | Helped |
| Rectify the coupling | MaskGIT · ImageNet 4-step | FID: 7.58 vs. 6.20 | Mixed |
| Apply ReDi rectification to DUO+DCD | DUO+DCD · OpenWebText 16-step | generative perplexity: 54.82 -> 30.34 | Helped |
| Apply ReDi rectification to Di4C | Di4C · OpenWebText 16-step | generative perplexity: 27.66 -> 24.81 | Helped |
| Use perturbed instead of standard rectification (ablation) | DUO+DCD · OpenWebText | generative perplexity: 44.17 -> 24.13 | Helped |
| Increase rectification pairs; performance saturates at 20k and fully saturates above 50k (ablation) | ReDi · ImageNet 4-step | FID | Helped |
| Change CFG used to sample rectification pairs (ablation) | ReDi^1 · ImageNet 4-step | FID: 9.76; 7.52; 7.77; 8.63 | Mixed |
| Lower the masked-state interpolation ratio r (ablation) | ReDi^0 · ImageNet 16-step | FID | Hurt |
| Lower the masked-state interpolation ratio r (ablation) | ReDi^1 coupling · ImageNet | Conditional TC | Helped |
| Iteratively rectify the coupling (ablation) | ReDi · ImageNet | Conditional TC | Helped |
| Use higher CFG for rectification-pair sampling (ablation) | ReDi^3-distill · ImageNet 1-step | FID: 14.11; 13.25; 11.68 | Helped |
Surprising findings
- Performance of the rectified DFM degrades in practice over successive rectification iterations due to error propagation, even though conditional TC decreases.
- The optimal CFG for rectification differs from the optimal CFG for distillation.
Limitations
- Connection between discrete rectification and continuous Rectified Flow is not fully elucidated.
- Work focuses on non-autoregressive DFMs; extension to autoregressive models is left open.
- Performance degrades over iterations due to error propagation from model-generated pairs.
Experimental setup
Datasets: ImageNet OpenWebText
Benchmarks: ImageNet OpenWebText
Metrics: FID IS precision recall density coverage generative perplexity entropy conditional TC
Baselines: MaskGIT SDTT Di4C DUO+DCD
Builds on: Rectified Flow Discrete Flow Matching MaskGIT DUO Di4C
Closest papers by meaning
- Discrete Flow Matching (NeurIPS 2024 · similarity 0.86)
- Improving the Training of Rectified Flows (NeurIPS 2024 · similarity 0.85)
- Text-to-Image Rectified Flow as Plug-and-Play Priors (ICLR 2025 · similarity 0.85)
- Flow Matching with General Discrete Paths: A Kinetic-Optimal Perspective (ICLR 2025 · similarity 0.85)
- Rectifying Categorical Flows on Statistical Manifolds for One-Step Generation (NeurIPS 2026 · similarity 0.84)
- Bernoulli Flow Models: Self-Consistent Generative Modeling for Binary Data (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.