Research map / Architectures and efficiency / Attention and transformers
Compound Word Transformer: Learning to Compose Full-Song Music over Dynamic Directed Hypergraphs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace REMI token-by-token modeling with compound words and type-specific heads | Linear Transformer for Pop piano generation · Conditional Pop piano | training time to NLL 0.27: 7 days -> 0.6 days | Helped |
| Replace REMI token-by-token modeling with compound words and type-specific heads | Linear Transformer for Pop piano generation · Conditional Pop piano | inference time per song: 48.9 -> 29.2 | Helped |
| Replace REMI token-by-token modeling with compound words and type-specific heads | Transformer-XL for Pop piano generation · Conditional Pop piano | inference time per song: 88.4 -> 29.2 | Helped |
| Use compound words instead of REMI tokens | CP + linear · Conditional Pop piano | melody matchness: 0.872 -> 0.829 | Hurt |
| Use compound words instead of REMI tokens | CP + linear · Conditional Pop piano | chord matchness: 0.785 -> 0.733 | Hurt |
| Use compound words instead of REMI tokens | CP + linear · Conditional Pop piano | subjective quality: Fidelity: 4.29 vs 4.09; Richness: 3.14 vs 3.13; Humanness: 3.70 vs 3.50; Correctness: 3.64 vs 3.31; Overall: 3.35 vs 3.0 | Mixed |
| Use compound words instead of REMI tokens | CP + linear · Unconditional Pop piano | inference time per song: 139.9 -> 19.8 | Helped |
| Use compound words instead of REMI tokens | CP + linear · Unconditional Pop piano | subjective quality: Fidelity: 3.11 -> 3.33; Humanness: 3.46 -> 3.68; Structureness: 2.91 -> 3.11; Overall: 3.03 -> 3.34 | Helped |
| Use compound words rather than individual tokens | Music sequence representation · Conditional training songs | mean sequence length: 6,432 (± 1,689) -> 3,142 (± 821) | Helped |
Closest papers by meaning
- Music Transformer: Generating Music with Long-Term Structure (ICLR 2019 · similarity 0.90)
- MuPT: A Generative Symbolic Music Pretrained Transformer (ICLR 2025 · similarity 0.90)
- N-gram Unsupervised Compoundation and Feature Injection for Better Symbolic Music Understanding (AAAI 2024 · similarity 0.89)
- Whole-Song Hierarchical Generation of Symbolic Music Using Cascaded Diffusion Models (ICLR 2024 · similarity 0.89)
- SongGen: A Single Stage Auto-regressive Transformer for Text-to-Song Generation (ICML 2025 · similarity 0.89)
- MIDI-GPT: A Controllable Generative Model for Computer-Assisted Multitrack Music Composition (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.