Research map / Image generation / Autoregressive and latent generation
MFlowAudio: Efficient Text-to-Audio Synthesis via Mamba-based Stateful Flow Matching
Hao Dai, Panyu Chen, Jagmohan Chauhan
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MFlowAudio, a latent text-to-audio generator that combines flow matching with a custom Mamba-based (TFMamba) backbone to avoid the quadratic cost of self-attention in long-form audio synthesis.
Problem
Transformer-based diffusion models for audio generation suffer from quadratic self-attention complexity, which bottlenecks long-form text-to-audio synthesis.
Approach
Pairs continuous-time flow matching in latent space with a custom TFMamba block (state-space model) and a stateful design, replacing self-attention to achieve efficient long-form audio generation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace Transformer backbone with TCMamba (Mamba-based) under similar size | MFlowAudio-base vs TangoFlux-base · AudioCaps 10s | FD: 16.9351 -> 10.9178 | Improved |
| use TCMamba instead of Transformer | MFlowAudio vs TangoFlux · AudioCaps 10s | CLAP and IS: IS 12.5489 -> 11.6865; CLAP 0.4990 -> 0.4957 | Hurt |
| use Mamba backbone with adaptive-length compute | MFlowAudio · generation latency 10s | latency (s): 3.51 -> 1.93 | Improved |
| add state propagation via SFM chunked generation | MFlowAudio · AudioCaps | FD: 11.8569 -> 11.5988 (chunk 64) | No clear effect |
| remove state propagation (overlap-add) in prompt switching (ablation) | MFlowAudio · dynamic prompt switching | BBR: 4.3287 -> 7.4001 | Hurt |
| add SDCG vs vanilla conditional field (ablation) | MFlowAudio · dynamic prompt switching | FD: 16.0940 -> 15.8575 | Improved |
| add SDCG vs vanilla conditional field (ablation) | MFlowAudio · dynamic prompt switching | BBR: 4.2633 -> 4.3287 | Hurt |
| add SDCG vs naive subtraction (ablation) | MFlowAudio · dynamic prompt switching | FD: 16.0056 -> 15.8575 | Improved |
| add stateful dynamic prompting vs TangoFlux stateless prompt switching | MFlowAudio · dynamic prompt switching | BBR: 11.6443 -> 4.3287 | Improved |
| use 8-frame overlap in VAE chunk decoding (ablation) | Oobleck VAE · chunk decoding | rel-L1: 4.54% -> 0.049% | Improved |
| increase overlap from 8 to 16 frames (ablation) | Oobleck VAE · chunk decoding | rel-L1: 0.049% -> 0.047% | No clear effect |
| stream with chunk size 64 vs one-time generation, 30s batch 2 | MFlowAudio · latency | time (s): 5.1 -> 1.85 | Improved |
Experimental setup
Builds on: flow matching Mamba
Closest papers by meaning
- UniFlow-Audio: Unified Flow Matching for Audio Generation from Omni-Modalities (NeurIPS 2026 · similarity 0.92)
- AudioCALM: Continuous Autoregressive Language Modeling for Universal Audio Generation (NeurIPS 2026 · similarity 0.89)
- BinauralFlow: A Causal and Streamable Approach for High-Quality Binaural Speech Synthesis with Flow Matching Models (ICML 2025 · similarity 0.88)
- Unified Synthesis of Compositional Speech and Sound from Free-Form Text Prompts (NeurIPS 2026 · similarity 0.88)
- ETTA: Elucidating the Design Space of Text-to-Audio Models (ICML 2025 · similarity 0.88)
- MusicFlow: Cascaded Flow Matching for Text Guided Music Generation (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
