Research map / Image generation / Autoregressive and latent generation
MIDI-GPT: A Controllable Generative Model for Computer-Assisted Multitrack Music Composition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Presents MIDI-GPT, a GPT2-based (~20M parameter) multitrack music generation system with a track-sequential tokenization, bar-level infilling, and attribute controls (instrument, density, polyphony, duration, style), released with an Open RAIL-M license.
Problem
Existing symbolic music generation systems are restricted in number of tracks, instrument schema, and infilling granularity, limiting their usability in computer-assisted composition workflows.
Approach
Represent each track as its own time-ordered token sequence concatenated across tracks (instead of interleaving tracks), decoupling track info from note tokens so any of 128 GM instruments can be used; add a Bar-Fill representation for bar-level infilling and attribute control tokens placed after the INSTRUMENT token, plus an expressive extension with velocity and microtiming DELTA tokens.
Key results
- Duplication of training excerpts decreases as the number of infilled bars grows, with near-duplicate rates falling significantly from 1 to 8 bars.
- At temperature 1.0, infilled generations are statistically equivalent to original material in StyleRank similarity.
- Note density control yields absolute differences mostly 0 and rarely above 1; note duration control meets the 70% in-range target in most cases, while polyphony control is less effective.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Generate longer infills | MIDI-GPT · GigaMIDI test-split infilling | frequency of near-duplicate training material | Helped |
| Generate 4 or more bars instead of shorter infills | MIDI-GPT · GigaMIDI test-split infilling | originality | Helped |
| Use temperature 1.0 | MIDI-GPT · GigaMIDI infilling | StyleRank similarity to the dataset | No clear effect |
| Increase temperature above 1.0 | MIDI-GPT · GigaMIDI infilling | StyleRank similarity to the dataset | Hurt |
| Condition generation on note density | MIDI-GPT · 8-bar generation from scratch | absolute difference between requested and generated note-density level | Helped |
| Condition generation on note-duration range | MIDI-GPT · 8-bar generation from scratch | percentage of note durations within the requested range | Mixed |
| Condition generation on polyphony range | MIDI-GPT · 8-bar generation from scratch | percentage of polyphony values within the requested range | Hurt |
Surprising findings
- Polyphony-level control is less effective than note-duration control, with median in-range outcomes below 70% in many cases.
Limitations
- Attention window of 2048 tokens limits generation to roughly 8-16 bars depending on track count and density.
- Training data biased toward pop and under-represents non-Western styles.
- Polyphony control is fuzzy and less reliable than duration control.
Experimental setup
Datasets: GigaMIDI MetaMIDI
Benchmarks: GigaMIDI test split
Metrics: Hamming distance Jaccard index StyleRank similarity binomial test percentage of values in attribute range absolute density difference
Baselines: MuseNet FIGARO CoCoNet MusicVAE MMT
Builds on: GPT-2 Pop Music Transformer MidiTok StyleRank MMM
Closest papers by meaning
- MuPT: A Generative Symbolic Music Pretrained Transformer (ICLR 2025 · similarity 0.90)
- MuseControlLite: Multifunctional Music Generation with Lightweight Conditioners (ICML 2025 · similarity 0.89)
- Unifying Symbolic Music Arrangement: Track-Aware Reconstruction and Structured Tokenization (NeurIPS 2025 · similarity 0.89)
- Text2midi: Generating Symbolic Music from Captions (AAAI 2025 · similarity 0.88)
- Structured Multi-Track Accompaniment Arrangement via Style Prior Modelling (NeurIPS 2024 · similarity 0.88)
- MID-FiLD: MIDI Dataset for Fine-Level Dynamics (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.