Research map / Image generation / Autoregressive and latent generation
Unifying Symbolic Music Arrangement: Track-Aware Reconstruction and Structured Tokenization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a unified segment-level reconstruction framework fine-tuning an 80M GPT-2-style symbolic music model across band arrangement, piano reduction, and drum arrangement, plus a track-wise multitrack tokenization called REMI-z.
Problem
Automatic multitrack symbolic music arrangement (reinterpretation, reduction, additive generation) has been handled by task-specific models that need parallel data or fixed mappings, and pre-trained symbolic music models are underused for fine-grained conditional arrangement.
Approach
A self-supervised segment-level reconstruction objective over token-level disentangled instrument, content, and history streams lets one pre-trained model perform any-to-any arrangement without parallel data, combined with REMI-z, a track-first tokenization that reduces content fragmentation and sequence length.
Key results
- Band arrangement: Ours reaches Note F1 97.5% and Note_i F1 87.0% vs 49.5% and 40.0% for Transformer-VAE.
- Piano reduction: Ours achieves F1 85.5 vs 58.3 for UNet, with higher human ratings for faithfulness, coherence, and playability.
- REMI-z reduces tokens per bar from 225.91 to 151.68 (32.9%) and note-level perplexity from 116.20 to 84.11 versus REMI+.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace REMI+ with REMI-z tokenization | Transformer band arrangement model · Band arrangement | Note F1: +6.7% | Helped |
| Add generative pre-training (ablation) | Unified arrangement model · Band arrangement | Note_i F1: +9.5% | Helped |
| Add voice ordering to instrument conditions (ablation) | Unified arrangement model · Band arrangement | VER: VER: +10.0% | Helped |
| Add voice ordering to instrument conditions (ablation) | Unified arrangement model · Band arrangement | Note_i F1: -2.7% | Helped |
| Add history conditioning (ablation) | Unified arrangement model · Band arrangement | Note_i F1: -9.6% | Helped |
| Add generative pre-training (ablation) | Unified arrangement model · Band arrangement | Faithfulness: -0.58 | Helped |
| Add generative pre-training (ablation) | Unified arrangement model · Piano reduction | F1: 78.4 -> 85.5 | Helped |
| Add generative pre-training (ablation) | Unified arrangement model · Drum arrangement | F1: 1.2 -> 79.3 | Helped |
| Replace REMI+ with REMI-z tokenization | Unconditional music generation model · Slakh2100 unconditional generation | Note-level perplexity: 84.11 vs. 116.20 | Helped |
| Replace REMI+ with REMI-z tokenization | Unconditional music generation model · Slakh2100 unconditional generation | Token-level perplexity: 4.50 vs. 3.00 | Hurt |
| Add octave shifts during training (ablation) | UNet piano reduction baseline · Piano reduction | Note-level F1: 58.31 to 42.80 | Hurt |
Surprising findings
- REMI+ achieves lower token-level perplexity (3.00) than REMI-z (4.50), while REMI-z wins on note-level perplexity.
Limitations
- Piano reduction covers only accompaniment (melody delegated), not full solo piano reduction.
- Velocity and tempo/time-signature tokens are excluded and fine-tuning is restricted to 4/4 songs.
- Human evaluation involves a small number of songs (5-6 per task) and 26 participants.
Experimental setup
Datasets: Los Angeles MIDI dataset Slakh2100 Lakh MIDI Dataset
Benchmarks: Slakh2100
Metrics: Note F1 Note_i F1 Mel F1 I-IoU VER perplexity token entropy 5-point human ratings (faithfulness, coherence, creativity,
Baselines: Transformer-VAE UNet Composer's Assistant 2 Rule-Based Rule-F Rule-O REMI+
Builds on: REMI+ REMI Composer's Assistant 2 GPT-2
Closest papers by meaning
- Structured Multi-Track Accompaniment Arrangement via Style Prior Modelling (NeurIPS 2024 · similarity 0.90)
- MuPT: A Generative Symbolic Music Pretrained Transformer (ICLR 2025 · similarity 0.89)
- MIDI-GPT: A Controllable Generative Model for Computer-Assisted Multitrack Music Composition (AAAI 2025 · similarity 0.89)
- Aria-MIDI: A Dataset of Piano MIDI Files for Symbolic Music Modeling (ICLR 2025 · similarity 0.88)
- Efficient Fine-Grained Guidance for Diffusion Model Based Symbolic Music Generation (ICML 2025 · similarity 0.88)
- Compose with Me: Collaborative Music Inpainter for Symbolic Music Infilling (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.