Research map / Image generation / Autoregressive and latent generation
Symbolic Music Generation with Non-Differentiable Rule Guided Diffusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Stochastic Control Guidance (SCG), a training-free plug-and-play method that steers pre-trained diffusion models with non-differentiable symbolic music rules by sampling multiple next-step realizations and selecting the one with lowest rule loss, plus a 10ms-resolution latent diffusion transformer for piano rolls.
Problem
Guiding pre-trained diffusion models for symbolic music generation under non-differentiable (and black-box) musical rules such as note density and chord progression, where gradient-based guidance fails.
Approach
Derives a path-integral stochastic control guidance that only requires forward evaluation of the rule loss: at each sampling step, draw several candidate next states, estimate clean samples via Tweedie's formula, and keep the candidate with the lowest rule loss, avoiding any gradient through the rule.
Key results
- On unconditional generation, the proposed model reaches average OA 0.943 on Maestro, 0.934 on Muscore and 0.939 on Pop, the highest among non-GT methods.
- For individual rule guidance on note density, SCG reduces loss to 0.131 versus 0.698 for classifier guidance and 1.261 for DPS-NN.
- For chord progression editing, SCG achieves 12.62% error versus 70.48% for PolyDiffusion.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use the proposed SCG music generator instead of MusicTr | unconditional symbolic music generation · Maestro | average OA: 0.903 ± 0.005 -> 0.943 ± 0.003 | Helped |
| use the proposed SCG music generator instead of MusicTr | unconditional symbolic music generation · Muscore | average OA: 0.901 ± 0.004 -> 0.934 ± 0.003 | Helped |
| use the proposed SCG music generator instead of CPW | unconditional symbolic music generation · Pop | average OA: 0.899 ± 0.005 -> 0.939 ± 0.004 | Helped |
| use SCG instead of classifier guidance | rule-guided generation · Note density | loss: 0.698 ± 0.587 -> 0.131 ± 0.325 | Helped |
| use SCG instead of DPS-NN | rule-guided generation · Chord progression | loss: 0.414 ± 0.256 -> 0.273 ± 0.1637 | Helped |
| use SCG instead of classifier guidance | rule-guided generation · Note density | OA: 0.861 ± 0.025 -> 0.842 ± 0.031 | Hurt |
| combine classifier guidance with SCG | composite rule generation · Composite rules | note density loss: 0.822 ± 0.844 -> 0.148 ± 0.203 | Helped |
| combine classifier guidance with SCG | composite rule generation · Composite rules | chord progression loss: 0.724 ± 0.205 -> 0.284 ± 0.197 | Helped |
| combine classifier guidance with SCG | composite rule generation · Composite rules | OA: 0.874 ± 0.007 -> 0.864 ± 0.010 | Hurt |
| increase the SCG sample count per step from 4 to 16 (ablation) | SCG note density guidance · Note density | loss: 0.318 ± 0.770 -> 0.131 ± 0.325 | Helped |
| increase classifier + SCG sample count per step from 4 to 16 (ablation) | classifier + SCG note density guidance · Note density | loss: 0.151 ± 0.298 -> 0.064 ± 0.159 | Helped |
| apply SCG every 2 steps instead of every step (ablation) | SCG note density guidance · Note density | loss: 0.131 ± 0.325 -> 0.365 ± 0.559 | Hurt |
Surprising findings
- DPS with the differentiable-rule loss fails on note density because its gradient is zero almost everywhere, yielding a loss of 2.508 and OA of 0.800.
- Applying SCG earlier in the sampling trajectory is less effective than applying it during early-to-mid phases, and SCG on every step gives lower loss than sparse application, with trade-offs against OA.
Limitations
- Sampling cost grows with the number of candidate samples per step (e.g., 1242.6 s for 16 samples vs 25.4 s without guidance), and OA drops relative to over-optimized control.
- Quantitative music quality evaluation relies on an OA proxy metric that the authors acknowledge is only a necessary, not sufficient, indicator.
- Composite guidance trades off the pitch histogram control and requires hand-tuned rule weights.
Experimental setup
Datasets: MAESTRO MuseScore (crawled) Pop1k7 POP909
Benchmarks: MAESTRO MuseScore Pop
Metrics: overlapping area (OA) L2 loss for pitch histogram and note density 0-1 loss for chord progression matching score editing error rate chroma and groove similarity subjective listening ratings
Baselines: MusicTransformer REMI CPW PolyDiff Classifier guidance DPS Figaro expert MuseCoCo MuseMorphose PolyDiffusion
Builds on: DDPM classifier guidance Diffusion Posterior Sampling DiT classifier-free guidance
Closest papers by meaning
- Efficient Fine-Grained Guidance for Diffusion Model Based Symbolic Music Generation (ICML 2025 · similarity 0.93)
- MuPT: A Generative Symbolic Music Pretrained Transformer (ICLR 2025 · similarity 0.87)
- Whole-Song Hierarchical Generation of Symbolic Music Using Cascaded Diffusion Models (ICLR 2024 · similarity 0.86)
- Unifying Symbolic Music Arrangement: Track-Aware Reconstruction and Structured Tokenization (NeurIPS 2025 · similarity 0.86)
- JEN-1 Composer: A Unified Framework for High-Fidelity Multi-Track Music Generation (AAAI 2025 · similarity 0.86)
- MIDI-GPT: A Controllable Generative Model for Computer-Assisted Multitrack Music Composition (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.