Research map / Image generation / Autoregressive and latent generation
YuE: Scaling Open Foundation Models for Long-Form Music Generation
Ruibin Yuan, Hanfeng Lin, Shuyue Guo, Ge Zhang, Jiahao Pan, Yongyi Zang, Haohe Liu, Yiming Liang, Wenye Ma, Xingjian Du, Xeron Du, Zhen Ye and 46 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Dual-NTP (separate vocal and accompaniment tokens) instead of standard NTP (ablation) | YuE 0.5B LM · training loss, 20B tokens | training loss: approximately 0.4 lower loss | Improved |
| add structural progressive conditioning (CoT) versus vanilla, curriculum, or RoPE base change (ablation) | YuE 0.5B LM · lyrics-following WER over 30-150 s | WER | Improved |
| scale model from 0.5B to 7B (ablation) | YuE · lyrics-following WER | WER: approximately 70% -> around 20% | Improved |
| increase RoPE base from 10K to 100K (ABF) or use curriculum on duration (ablation) | YuE 0.5B LM · lyrics-following WER | WER | No clear effect |
| add CFG on top of ICL at inference (ablation) | YuE-7B · musicality A/B win rate | win rate: 0.63 -> 0.79 | Improved |
| use ICL reference audio instead of CoT only at inference (ablation) | YuE-7B · musicality A/B win rate | win rate: 0.21 -> 0.63 | Improved |
| use semantic-acoustic fused tokenizer (X-Codec) instead of pure acoustic codecs (ablation) | YuE LM · LM convergence on in-the-wild music | LM convergence | Improved |
| introduce ICL data early in pre-training rather than only during annealing (ablation) | YuE · pre-training with ICL | creativity / lyric control | Hurt |
| increase TTS data proportion excessively in multitask mix (ablation) | YuE · non-rap genres | singing quality | Hurt |
| YuE vs CLAP score as automated alignment metric correlated with human lyric following | CLaMP 3 vs CLAP · correlation with LyricFollow | Pearson correlation: CLaMP 3 0.42 vs CLAP -0.25 | Mixed |
| vocal range as proxy metric for musicality | automatic metrics · correlation with human musicality | Pearson correlation: 0.857 | Improved |
Closest papers by meaning
- SongGen: A Single Stage Auto-regressive Transformer for Text-to-Song Generation (ICML 2025 · similarity 0.90)
- S²MILE: Semantic-and-Structure-Aware Music-Driven Lyric Generation (AAAI 2025 · similarity 0.89)
- SongGLM: Lyric-to-Melody Generation with 2D Alignment Encoding and Multi-Task Pre-Training (AAAI 2025 · similarity 0.89)
- Whole-Song Hierarchical Generation of Symbolic Music Using Cascaded Diffusion Models (ICLR 2024 · similarity 0.89)
- CSL-L2M: Controllable Song-Level Lyric-to-Melody Generation Based on Conditional Transformer with Fine-Grained Lyric and Musical Controls (AAAI 2025 · similarity 0.88)
- SongBloom: Coherent Song Generation via Interleaved Autoregressive Sketching and Diffusion Refinement (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
