Research map / Large language models / Language, safety and interpretability
AudioStory: Generating Long-Form Narrative Audio with Large Language Models
Yuxin Guo, Teng Wang, Yuying Ge, Shijie Ma, Yixiao Ge, Wei Zou
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add LLM-based instruction decomposition and decoupled bridging (AudioStory) versus LLM+TangoFlux captioning | TangoFlux · AS-10k long-form narrative audio | CLAP: 0.322 -> 0.392 | Improved |
| add storyline reasoning before generation (ablation) | AudioStory · AS-10k | instruction-following score: 3.1 -> 4.1 | Improved |
| add explicit per-clip captions (interleaved reasoning generation) (ablation) | AudioStory · AS-10k | FAD: 16.03 -> 3.06 | Improved |
| supervise semantic tokens with T5 text instead of AudioMAE/Whisper audio features (ablation) | AudioStory bridging · Single audio generation | bridging ablation score: 9.55 -> 2.29 (AudioMAE vs T5) | Improved |
| supervise residual tokens weakly via DiT flow loss instead of explicit audio-feature supervision (ablation) | AudioStory bridging · Single audio generation | bridging ablation score: 3.71 -> 2.29 (Whisper-guided vs ours) | Improved |
| fully unfreeze DiT in joint training instead of freezing it (ablation) | AudioStory · Single audio generation | FAD: 11.04 -> 2.29 | Improved |
| unfreeze all DiT rather than only Single-DiT or only MM-DiT (ablation) | AudioStory · Single audio generation | FAD: 5.84 / 3.21 -> 2.29 | Improved |
| add residual tokens to the bridge in joint training (ablation) | AudioStory · Single audio generation | FAD: 4.66 -> 2.29 | Improved |
| train generation first then understanding (stage order G then U) instead of understanding first (ablation) | AudioStory · Long-form audio generation | FAD: 9.53 -> 3.00 | Improved |
| add DiT joint training in Stage-II before understanding stage versus after (ablation) | AudioStory · Long-form audio generation | FAD: 6.50 -> 3.00 | Improved |
| use progressive training instead of joint training in one stage (ablation) | AudioStory · Long-form audio generation | FAD: 8.74 -> 3.00 | Improved |
| AudioStory-Base versus TangoFlux on single-audio CLAP | TangoFlux · AudioCaps test | CLAP: 0.385 -> 0.383 | No clear effect |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- TangoFlux — TangoFlux: Super Fast and Faithful Text to Audio Generation with Flow Matching and Clap-Ranked Preference Optimization (ICLR 2026) · better on 3 of 4 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- ThinkSound: Chain-of-Thought Reasoning in Multimodal LLMs for Audio Generation and Editing (NeurIPS 2025 · similarity 0.89)
- Audio Flamingo 2: An Audio-Language Model with Long-Audio Understanding and Expert Reasoning Abilities (ICML 2025 · similarity 0.89)
- AudioCALM: Continuous Autoregressive Language Modeling for Universal Audio Generation (NeurIPS 2026 · similarity 0.88)
- Unified Synthesis of Compositional Speech and Sound from Free-Form Text Prompts (NeurIPS 2026 · similarity 0.88)
- MM-Narrator: Narrating Long-form Videos with Multimodal In-Context Learning (CVPR 2024 · similarity 0.88)
- ETTA: Elucidating the Design Space of Text-to-Audio Models (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
