Research map / Image generation / Autoregressive and latent generation
From Text to Talk: Audio-Language Model Needs Non-Autoregressive Joint Training
Tianqiao Liu, Xueyi Li, Hao Wang, Haoxuan Li, Zhichao Chen, Weiqi Luo, Zitao Liu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add batchwise AR & NAR objective mixing (BANOM) (ablation) | TtT-3B · Audio-QA LQ. | score: 19.87 -> 34.68 | Improved |
| add prefix preservation masking (PPM) (ablation) | TtT-3B · Audio-QA LQ. | score: 22.79 -> 34.68 | Improved |
| add stochastic span truncation (SST) (ablation) | TtT-3B · Audio-QA LQ. | score: 10.20 -> 34.68 | Improved |
| add stochastic span truncation (SST) (ablation) | TtT-3B · ASR AISHELL-2 | WER: 25.43 -> 12.53 | Improved |
| add stochastic span truncation (SST) (ablation) | TtT-3B · ASR AISHELL-1 | WER: 31.03 -> 13.65 | Improved |
| add batchwise AR & NAR objective mixing (BANOM) (ablation) | TtT-3B · ASR AISHELL-2 | WER: 18.58 -> 12.53 | Improved |
| add prefix preservation masking (PPM) (ablation) | TtT-3B · ASR AISHELL-2 | WER: 15.63 -> 12.53 | Improved |
| add stochastic span truncation (SST) (ablation) | TtT-3B · ASR Fen. | WER: 62.60 -> 64.31 | Hurt |
| replace AR audio generation with hybrid AR text + NAR audio diffusion | Qwen2.5-3B · Audio-QA LQ. | score: 10.00 -> 34.68 | Improved |
| replace AR audio generation with hybrid AR text + NAR audio diffusion | Qwen2.5-3B · ASR AISHELL-2 | WER: 54.94 -> 12.53 | Improved |
| use fully NAR generation for text and audio instead of AR text | Qwen2.5-3B · ASR AISHELL-2 | WER: 54.94 -> 212.27 | Hurt |
| add large-scale multimodal pretraining before TtT training | TtT-3B · Audio-QA LQ. | score: 34.68 -> 40.07 | Improved |
Closest papers by meaning
- DELTA-TTS: Adapting Autoregressive Model into a Diffusion Language Model for Text-to-Speech (NeurIPS 2026 · similarity 0.92)
- AudioCALM: Continuous Autoregressive Language Modeling for Universal Audio Generation (NeurIPS 2026 · similarity 0.91)
- Generative Audio Language Modeling with Continuous-valued Tokens and Masked Next-Token Prediction (ICML 2025 · similarity 0.90)
- Measuring Audio's Impact on Correctness: Audio-Contribution-Aware Post-Training of Large Audio Language Models (ICLR 2026 · similarity 0.88)
- Unified Synthesis of Compositional Speech and Sound from Free-Form Text Prompts (NeurIPS 2026 · similarity 0.88)
- Scaling Speech-Text Pre-training with Synthetic Interleaved Data (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
