Research map / Probabilistic and scientific modeling / Variational and generative sampling
Minibatch Optimal Transport and Perplexity Bound Estimation in Discrete Flow Matching
Etrit Haxholli, Yeti Z. Gurbuz, Oğul Can, Eli Waxman
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add minibatch OT (Sinkhorn) to BoW discrete flow (ablation) | DFM-B · OWT generative perplexity, 1024 steps | generative perplexity: 185.12 -> 178.53 | Improved |
| add minibatch OT (Sinkhorn) to multimask flow (ablation) | DFM-MMF · OWT generative perplexity, 1024 steps | generative perplexity: 143.48 -> 141.92 | Improved |
| replace Sinkhorn with exact OT solver (ablation) | DFM-MMF · OWT generative perplexity, 1024 steps | generative perplexity: 141.92 -> 141.46 | No clear effect |
| compute OT couplings with EMA embeddings (OT-EMA) on top of exact OT (ablation) | DFM-MMF-Exact · OWT generative perplexity, 1024 steps | generative perplexity: 141.46 -> 132.55 | Improved |
| compute OT couplings with EMA embeddings (OT-EMA) on top of exact OT (ablation) | DFM-B-Exact · OWT generative perplexity, 8 steps | generative perplexity: 335.14 -> 302.76 | Improved |
| use multimask flow instead of single-mask source (ablation) | masked DFM · OWT generative perplexity, 1024 steps | generative perplexity: 147.07 -> 143.48 | Improved |
| use BoW source instead of multimask source (ablation) | DFM with OT · OWT generative perplexity, few vs many steps | generative perplexity: 8 steps: 302.76 vs 480.61; 1024 steps: 159.87 vs 132.55 | Mixed |
| add minibatch OT to Morse Shakespeare flow | GPT-2 DFM, BoW source · Shakespeare Morse-code, L=128 | jumps: 85.47 -> 74.87 | Improved |
| increase Sinkhorn entropy regularization epsilon (ablation) | minibatch OT · Shakespeare Morse-code, L=128 | jumps: 74.87 -> 82.86 | Hurt |
| reduce number of mask tokens to 256 (ablation) | DFM-MMF with OT · OWT | generative perplexity | Hurt |
| increase OT batch size to 4096 (ablation) | minibatch OT · OWT, 32 and 64 steps | generative perplexity | Improved |
| train with the (1-t)-weighted bound (DFM-N) rather than MD4-style loss (DFM-S) (ablation) | masked DFM · OWT generative perplexity, 8 steps | generative perplexity: 587.80 -> 556.73 | Improved |
Closest papers by meaning
- Discrete Flow Matching (NeurIPS 2024 · similarity 0.91)
- Optimal Flow Matching: Learning Straight Trajectories in Just One Step (NeurIPS 2024 · similarity 0.90)
- Flow matching achieves almost minimax optimal convergence (ICLR 2025 · similarity 0.90)
- On the Relation between Rectified Flows and Optimal Transport (NeurIPS 2025 · similarity 0.89)
- Categorical Flow Matching on Statistical Manifolds (NeurIPS 2024 · similarity 0.89)
- Flow Matching with General Discrete Paths: A Kinetic-Optimal Perspective (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
