Research map / Probabilistic and scientific modeling / Molecules and proteins
Adaptive Accompaniment with ReaLchords
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Finetunes an online MLE chord-accompaniment transformer with KL-regularized RL using self-supervised contrastive/discriminative coherence rewards and on-policy distillation from a non-causal offline teacher that sees the full melody.
Problem
Online accompaniment requires generating chords simultaneously with a user's melody without seeing future notes; MLE-trained models suffer exposure bias and fail to adapt to mistakes or key changes.
Approach
Combine RL finetuning with self-supervised multi-scale contrastive and discriminative melody-chord reward models, plus a KL distillation term toward an offline MLE teacher that sees the full melody, forcing the causal student to anticipate.
Key results
- RLChords-M reaches 54.29% note-in-chord ratio versus 36.99% for Online MLE and 63.73% for Offline MLE.
- In a listening test with 10 musicians, both RLChords and RLChords-M significantly outperformed KD, and all systems outperformed Online MLE.
- In the perturbation experiment, Online MLE never recovers after a tritone transposition at beat 17, whereas RLChords and RLChords-M quickly adapt.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Distill from the offline teacher (ablation) | Online MLE accompaniment · Hooktheory test set | Note in Chord (%): 36.99 -> 46.38 | Helped |
| Use a contrastive reward model (ablation) | Online MLE accompaniment · Hooktheory test set | Note in Chord (%): 36.99 -> 47.22 | Helped |
| Use a discriminative reward model (ablation) | Online MLE accompaniment · Hooktheory test set | Δ Chord-Note Onset Interval (× 10^-3): 14.23 -> 12.34 | Helped |
| Combine contrastive and discriminative rewards (ablation) | Online MLE accompaniment · Hooktheory test set | Note in Chord (%): 36.99 -> 46.12 | Helped |
| Add RL finetuning with rewards and distillation | Online MLE accompaniment · Hooktheory test set | Note in Chord (%): 36.99 -> 48.17 | Helped |
| Use multi-scale reward models instead of global reward models (ablation) | ReaLchords · Hooktheory test set | Note in Chord (%): 48.17 -> 54.29 | Helped |
| Remove the repetition penalty (ablation) | ReaLchords · Hooktheory test set | Long Chords Ratio (%): 0.02 -> 0.78 | Hurt |
| Remove the silence penalty in addition to the repetition penalty (ablation) | ReaLchords · Hooktheory test set | Chord Silence Ratio (%): 4.90 -> 26.50 | Hurt |
| Remove all regularization penalties (ablation) | ReaLchords · Hooktheory test set | Chord Silence Ratio (%): 4.90 -> 27.85 | Hurt |
| Remove all regularization penalties (ablation) | Knowledge Distillation (KD) · Hooktheory test set | Early Stop Ratio (%): 0 -> 0.38 | Hurt |
| Use policy samples rather than dataset, mixed, or teacher-inclusive samples for KD (ablation) | Knowledge distillation · Hooktheory test set | Δ Chord-Note Onset Interval (× 10^-3): 19.58 -> 12.81 | Helped |
| Add distillation and both reward models | ReaLchords · Hooktheory test set | Chord Length Entropy: 1.56 -> 1.35 | Hurt |
Surprising findings
- RL finetuning induces a 'wait and see' behavior where models stay silent in the first beats before committing to chords.
- RL-Duet-style setups using offline MLE or online and offline MLE as reward models collapsed to repeating a single chord.
Limitations
- Training assumes chords do not influence the melody (one-way accompaniment), a simplification of full joint improvisation.
- Evaluation relies on a small listening test of 10 musicians and heuristic metrics; RLChords-M vs RLChords difference is not significant.
- Reward models are self-supervised and not validated against human labels beyond correlation with a listening test.
Experimental setup
Datasets: Hooktheory (updated, 38K melody-chord pairs)
Benchmarks: Hooktheory test set
Metrics: note-in-chord ratio chord-note onset interval EMD chord length entropy pairwise listening test preference (5-point Likert) Kruskal-Wallis H test Wilcoxon signed-rank with Bonferroni correction
Baselines: Online MLE Offline MLE Knowledge Distillation (KD) RL-Duet SongDriver
Builds on: RLHF on-policy knowledge distillation RL-Duet REINFORCE T5
Closest papers by meaning
- Live Music Diffusion Models: Efficient Fine-Tuning and Post-Training of Interactive Diffusion Music Generators (NeurIPS 2026 · similarity 0.88)
- Responding to the Call: Exploring Automatic Music Composition Using a Knowledge-Enhanced Model (AAAI 2024 · similarity 0.86)
- Efficient Fine-Grained Guidance for Diffusion Model Based Symbolic Music Generation (ICML 2025 · similarity 0.86)
- MusicRL: Aligning Music Generation to Human Preferences (ICML 2024 · similarity 0.85)
- JEN-1 Composer: A Unified Framework for High-Fidelity Multi-Track Music Generation (AAAI 2025 · similarity 0.85)
- Multi-Source Diffusion Models for Simultaneous Music Generation and Separation (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.