Research map / Large language models / Alignment and preferences
LeVo: High-Quality Song Generation with Multi-Preference Alignment
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
LeVo is a language-model song generator that predicts mixed and dual-track audio tokens in parallel via a two-decoder LeLM with modular extension training, then aligns it with multi-preference DPO via interpolated per-preference models.
Problem
Lyrics-to-song generation struggles with vocal-instrument harmony, audio quality, musicality, and instruction following, partly due to limited vocabulary of mixed tokens, interference between token types, and scarce high-quality annotated data.
Approach
Parallel modeling of mixed tokens (LM) and dual-track vocal/accompaniment tokens (AR decoder fed with LM hidden states) trained in stages to avoid interference, plus DPO on three preference dimensions (lyric alignment, prompt consistency, musicality) merged by parameter interpolation.
Key results
- LeVo achieves the lowest PER (7.2%) among all systems including industry tools and highest MuQ-T (0.34), and highest Audiobox-Aesthetic CE, CU, PQ.
- In subjective MOS, LeVo scores 3.38 on LYC, exceeding Suno-V4.5 (3.17), and outperforms all open-source academic systems on all six dimensions.
- Removing DPO raises PER from 7.2% to 10.6% and lowers OVL from 3.42 to 3.18 in MOS; interpolation of three preference-specific models outperforms mixed training on most metrics.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add modular extension training (Stage 2) (ablation) | LeVo · Song generation | PER: 17.5 -> 7.2 | Helped |
| add modular extension training (Stage 2) (ablation) | LeVo · Song generation | LYC MOS: 2.91 -> 3.38 | Helped |
| add the AR decoder (ablation) | LeVo · Song generation | PER: 26.0 -> 7.2 | Helped |
| add the AR decoder (ablation) | LeVo · Song generation | LYC MOS: 2.74 -> 3.38 | Helped |
| add dual-track token prediction (ablation) | LeVo · Song generation | PER: 11.0 -> 7.2 | Helped |
| add dual-track token prediction (ablation) | LeVo · Song generation | LYC MOS: 3.18 -> 3.38 | Helped |
| add DPO-based multi-preference alignment (ablation) | LeVo · Song generation | PER: 10.6 -> 7.2 | Helped |
| add DPO-based multi-preference alignment (ablation) | LeVo · Song generation | LYC MOS: 3.18 -> 3.38 | Helped |
| add DPO-based multi-preference alignment (ablation) | LeVo · Song generation | FAD: 2.60 -> 2.68 | Hurt |
| fine-tune with Lyric Alignment Preference (Strategy 1) (ablation) | LeVo DPO · Song generation | PER: 10.6 -> 6.5 | Helped |
| fine-tune with Prompt Consistency Preference (Strategy 2) (ablation) | LeVo DPO · Song generation | MuQ-T: 0.31 -> 0.34 | Helped |
| fine-tune with Prompt Consistency Preference (Strategy 2) (ablation) | LeVo DPO · Song generation | MuQ-A: 0.82 -> 0.83 | Helped |
Surprising findings
- Lyric-alignment-only DPO (Strategy 1) reduced PER to 6.5% but increased FAD, showing trade-offs across preference dimensions.
Limitations
- Audio quality remains constrained by variable training data quality and discrete tokens, leaving a gap to industry models.
- Reliance on pseudo-labels from Qwen2-Audio and other pre-trained tools limits instruction-following fidelity and prompt diversity.
- Remaining disparity with Suno on several subjective metrics and song structure clarity.
Experimental setup
Datasets: in-house 2M-song music dataset (~110k hours) DPO preference pairs (lyric, prompt, musicality)
Benchmarks: in-house song evaluation set (lyrics and prompts)
Metrics: FAD MuQ-T MuQ-A PER Audiobox-Aesthetic CE/CU/PC/PQ MOS OVL/MEL/HAM/SSC/AQ/LYC
Baselines: Suno-V4.5 Mureka-O1 Haimian YuE DiffRhythm ACE-Step SongGen
Builds on: DPO MuCodec SongGen YuE DNI (Deep Network Interpolation)
Closest papers by meaning
- SongGen: A Single Stage Auto-regressive Transformer for Text-to-Song Generation (ICML 2025 · similarity 0.90)
- CMI-RewardBench: Evaluating Music Reward Models with Compositional Multimodal Instruction (ICML 2026 · similarity 0.89)
- SongGLM: Lyric-to-Melody Generation with 2D Alignment Encoding and Multi-Task Pre-Training (AAAI 2025 · similarity 0.89)
- SongCreator: Lyrics-based Universal Song Generation (NeurIPS 2024 · similarity 0.88)
- MusicRL: Aligning Music Generation to Human Preferences (ICML 2024 · similarity 0.88)
- CSL-L2M: Controllable Song-Level Lyric-to-Melody Generation Based on Conditional Transformer with Fine-Grained Lyric and Musical Controls (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.