Research map / Image generation / Autoregressive and latent generation

Kinetic-Optimal Scheduling with Moment Correction for Metric-Induced Discrete Flow Matching in Zero-Shot Text-to-Speech

NeurIPS 2026 · Image generation › Autoregressive and latent generation · approach: Neural text-to-speech synthesis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes GibbsTTS: a training-free constant-Fisher-Rao-speed scheduler and a finite-step moment correction for metric-induced discrete flow matching, applied to codec-token zero-shot TTS.

Problem

Metric-induced discrete flow matching (MI-DFM) relies on heuristic time schedulers needing hyperparameter search and suffers finite-step path-tracking error from a first-order CTMC solver, limiting its use in zero-shot TTS.

Approach

Derives a kinetic-optimal (constant Fisher-Rao speed) scheduler for prescribed scalar-parameterized paths and computes it numerically from token-embedding distance matrices, plus a finite-step moment correction that adjusts only the jump probability while keeping the jump destination distribution fixed.

Key results

• GibbsTTS (Numerical KO + corrector) reaches UTMOS 3.651, WER 1.777% and SIM 0.743 on Seed-TTS test-en, best among masked generative baselines.
• Removing the moment corrector lowers UTMOS on Seed-TTS test-en from 3.651 to 3.403 and raises WER from 1.777% to 2.120%.
• GibbsTTS attains the highest speaker similarity on three of four SOTA comparison test sets (e.g., 0.790 on Seed-TTS test-zh), second on CosyVoice 3 en behind OmniVoice.

Surprising findings

• The MaskGCT sin scheduler performs poorly with masked DFM but is strongest among masked DD schedulers, showing scheduler choice is coupled to decoding formulation.
• GibbsTTS is not the strongest system on naturalness or WER/CER against larger AR and recent NAR systems, so SOTA gains are mainly in speaker similarity.

Limitations

• Distance is restricted to cosine-equivalent codebook distance; other token metrics and geometry not explored.
• Corrector uses an unconditional reference moment, so it is an approximation and does not recover the infinitesimal CTMC limit as h→0.
• Only evaluated on zero-shot TTS; SOTA comparisons are not fully controlled and GibbsTTS trails on naturalness and WER/CER.

Experimental setup

Datasets: Emilia-en Emilia-zh LibriTTS test-clean Seed-TTS test-en Seed-TTS test-zh CosyVoice 3 test sets

Benchmarks: Seed-TTS test-en Seed-TTS test-zh CosyVoice 3 en CosyVoice 3 zh

Metrics: UTMOS WER CER SIM CMOS SMOS

Baselines: MI-DFM grid-searched scheduler Masked DFM closed-form KO DiFlow-TTS MaskGCT F5-TTS OmniVoice Spark-TTS FireRedTTS-2 IndexTTS2 CosyVoice 2 CosyVoice 3 Qwen3-TTS

Builds on: metric-induced discrete flow matching discrete flow matching MaskGCT kinetic-optimal discrete probability paths DiT

Closest papers by meaning

• Trajectory as the Teacher: Few-Step Discrete Flow Matching via Energy-Navigated Distillation (NeurIPS 2026 · similarity 0.87)
• DMOSpeech: Direct Metric Optimization via Distilled Diffusion Model in Zero-Shot Speech Synthesis (ICML 2025 · similarity 0.85)
• Shallow Flow Matching for Coarse-to-Fine Text-to-Speech Synthesis (NeurIPS 2025 · similarity 0.85)
• Discrete Flow Matching (NeurIPS 2024 · similarity 0.85)
• MaskGCT: Zero-Shot Text-to-Speech with Masked Generative Codec Transformer (ICLR 2025 · similarity 0.84)
• CLaM-TTS: Improving Neural Codec Language Model for Zero-Shot Text-to-Speech (ICLR 2024 · similarity 0.84)