Research map / Image generation / Autoregressive and latent generation
SongSong: A Time Phonograph for Chinese SongCi Music from Thousand of Years Away
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces SongSong, a multi-stage system that turns Chinese SongCi poems into full songs (melody, singing voice, accompaniment), plus the OpenSongSong dataset of 29.9 hours of SongCi music.
Problem
Existing music generation models target modern pop and cannot produce ancient Chinese SongCi music with its distinct rhythms and styles; there is no open dataset of ancient SongCi music.
Approach
A lyric-to-rhythm-to-melody Transformer pipeline followed by FastSpeech2-style singing voice synthesis with a diffusion F0 predictor and shallow diffusion mel decoder, plus a MusicGen-melody accompaniment module, all conditioned on a melody derived from the SongCi text.
Key results
- In zero-shot tests SongSong scored 75.37 pronunciation accuracy and 78.44 SongCi style conformity versus Suno's 62.59 and 35.30.
- SongSong obtained the lowest FAD_vgg (5.41) among the compared systems in the main table, while Suno and SkyMusic scored higher on common subjective metrics such as music structure and motivation continuity.
- Against GPT-SoVITS in the ablation, SongSong improved PA from 45.25 to 81.25 and SCS from 46.25 to 75.50.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SongSong instead of Suno or SkyMusic | SongCi music generation, zero-shot · SongCi test, zero-shot | SCS: 35.30 / 25.37 -> 78.44 | Helped |
| Use SongSong instead of Suno or SkyMusic | SongCi music generation, few-shot · SongCi test, few-shot | SCS: 42.19 / 46.67 -> 75.19 | Helped |
| Use SongSong instead of Suno or SkyMusic | SongCi music generation, zero-shot · SongCi test, zero-shot | PA: 62.59 / 54.81 -> 75.37 | Helped |
| Use SongSong instead of Suno or SkyMusic | SongCi music generation, few-shot · SongCi test, few-shot | PA: 48.85 / 59.04 -> 78.48 | Helped |
| Use SongSong instead of Suno or SkyMusic | SongCi music generation, zero-shot · SongCi test, zero-shot | VN: 65.37 / 72.89 vs 65.56 | Mixed |
| Use SongSong instead of Suno or SkyMusic | SongCi music generation, few-shot · SongCi test, few-shot | VN: 71.04 / 72.74 vs 72.81 | No clear effect |
| Use SongSong instead of Suno or SkyMusic | SongCi music generation, zero-shot · SongCi test, zero-shot | FAD_vgg: 5.74 / 7.88 -> 5.41 | Helped |
| Use SongSong instead of Suno or SkyMusic | SongCi music generation, few-shot · SongCi test, few-shot | FAD_vgg: 7.92 / 6.31 -> 5.41 | Helped |
| Replace SongSong's rhythm-based lyric-melody alignment with GSV-style speech synthesis architecture (ablation) | SongCi music generation · SongCi clips, ablation evaluation | FAD_vgg: 7.22 -> 20.30 | Hurt |
| Replace SongSong's rhythm-based lyric-melody alignment with GSV-style speech synthesis architecture (ablation) | SongCi music generation · SongCi clips, ablation evaluation | PA: 81.25 -> 45.25 | Hurt |
| Replace SongSong's rhythm-based lyric-melody alignment with GSV-style speech synthesis architecture (ablation) | SongCi music generation · SongCi clips, ablation evaluation | SCS: 75.50 -> 46.25 | Hurt |
Surprising findings
- Suno and SkyMusic scored higher than SongSong on common subjective metrics (music structure, equipment richness, motivation continuity), which the authors attribute to SongSong's fixed SongCi structure and monotony.
Limitations
- Evaluation uses only 85 utterances and 9 experts; small-scale subjective testing.
- Lower sound quality due to limited high-quality Song Dynasty training data.
- Lower scores than commercial systems on common music-quality metrics; the paper's FAD_vgg is also higher than Suno/SkyMusic in some settings.
Experimental setup
Datasets: OpenSongSong Opencpop M4Singer
Benchmarks: SongSong test set (85 utterances from 50 held-out songs)
Metrics: FAD_vgg FAD_pann music structure equipment richness motivation continuity sound quality pronunciation accuracy voice naturality SongCi style conformity accompaniment conformity
Baselines: Suno SkyMusic GPT-SoVITS
Builds on: FastSpeech2 DiffSinger MusicGen Encodec TeleMelody
Closest papers by meaning
- LeVo: High-Quality Song Generation with Multi-Preference Alignment (NeurIPS 2025 · similarity 0.85)
- FT-GAN: Fine-Grained Tune Modeling for Chinese Opera Synthesis (AAAI 2024 · similarity 0.83)
- Whole-Song Hierarchical Generation of Symbolic Music Using Cascaded Diffusion Models (ICLR 2024 · similarity 0.83)
- SongBloom: Coherent Song Generation via Interleaved Autoregressive Sketching and Diffusion Refinement (NeurIPS 2025 · similarity 0.82)
- MuPT: A Generative Symbolic Music Pretrained Transformer (ICLR 2025 · similarity 0.82)
- Rubato: Transcribing Piano Music with Timestamps (NeurIPS 2026 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.