Research map / Large language models / Language, safety and interpretability
Aria-MIDI: A Dataset of Piano MIDI Files for Symbolic Music Modeling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds Aria-MIDI, a dataset of over one million piano MIDI files transcribed from internet audio, using LLM-guided crawling, a source-separation-distilled audio classifier for piano detection and segmentation, and Aria-AMT transcription.
Problem
Symbolic piano MIDI datasets are small or low quality (or noisy when scraped), and curating large web-sourced audio for transcription without human oversight is hard.
Approach
A language model crawls and scores YouTube videos for solo-piano likelihood, an audio classifier distilled from a source-separation model's pseudo-labels prunes and segments recordings, and a Whisper-based AMT model transcribes them, with LLM-extracted metadata enabling compositional deduplication.
Key results
- Aria-MIDI contains 1,186,253 MIDI files totaling about 100,629 hours; 800,973 files remain after compositional deduplication.
- The proposed classifier with lambda=0.5 achieves 73.86% segmentation F1 on the full corpus versus 67.96% for MVSep at -22dB, and removes 100% of non-piano files at a file-level score threshold of 0.7 while retaining 95.28% of high-quality piano files.
- Aria-AMT achieves 97.63% note F1 on MAESTRO versus 96.82% for the GiantMIDI transcription model, and 90.58% vs 83.10% on MAPS.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace direct MVSep segmentation with the proposed audio classifier and segmentation pipeline | Piano audio segmentation · Full corpus | segmentation F1: 67.96 -> 73.86 | Helped |
| use λ=0.6 instead of λ=0.5 (ablation) | Proposed audio segmentation pipeline · Full corpus | segmentation F1: 73.86 -> 72.55 | Hurt |
| use λ=0.7 instead of λ=0.5 (ablation) | Proposed audio segmentation pipeline · Full corpus | segmentation F1: 73.86 -> 69.74 | Hurt |
| remove pseudo-labeled training data (ablation) | Piano audio classifier · Full corpus | segmentation F1: 73.86 -> 75.86 | Helped |
| remove pseudo-labeled training data (ablation) | Piano audio classifier · Full corpus | non-piano overlap: 98.83 -> 91.10 | Hurt |
| replace Llama 3.1 8B with Llama 3.1 70B | YouTube video classification · score ≥5 | F1: 41.30 -> 85.11 | Helped |
| replace Llama 3.1 70B with Llama 3.1 405B | YouTube video classification · score ≥4 | F1: 81.93 -> 86.84 | Helped |
| use Aria-AMT instead of the previous transcription model | Piano transcription · MAESTRO v3 test | note F1: 96.82 -> 97.63 | Helped |
| use Aria-AMT instead of the previous transcription model | Piano transcription · MAPS test | note F1: 83.10 -> 90.58 | Helped |
| raise the segment average-score threshold from T≥0.50 to T≥0.70 (ablation) | Audio classifier · All solo-piano recordings | classification F1: 96.17 -> 94.66 | Hurt |
Surprising findings
- Llama 3.1 405B with score threshold 4 is best, but the 8B model performs competitively with the 70B on some metrics, with 70B chosen for cost-accuracy balance.
- The ablation classifier without pseudo-labeled data has higher piano overlap but mislabels non-piano audio as piano about eight times more often.
Limitations
- Copyright restrictions mean audio and raw metadata are not released, only MIDI and derived labels.
- Metadata extraction and LLM classification show notable errors (e.g., 'Period' missed labels at 12.5%), and the dataset is dominated by classical music.
- Pipeline relies on YouTube metadata and recommendation algorithms, with the source-separation model access dependent on a third party.
Experimental setup
Datasets: MAESTRO AudioSet GiantMIDI audio Jazz Trio Database MAPS YouTube videos crawled via API
Benchmarks: MAESTRO v3 MAPS
Metrics: precision recall F1 segmentation accuracy segment overlap
Baselines: MVSep Piano GiantMIDI transcription model (high-resolution piano transcription) Lakh MAESTRO ATEPP PiJAMA
Builds on: Aria-AMT MVSep Piano source separation GiantMIDI Lakh MIDI dataset
Closest papers by meaning
- Rubato: Transcribing Piano Music with Timestamps (NeurIPS 2026 · similarity 0.91)
- Text2midi: Generating Symbolic Music from Captions (AAAI 2025 · similarity 0.89)
- Unifying Symbolic Music Arrangement: Track-Aware Reconstruction and Structured Tokenization (NeurIPS 2025 · similarity 0.88)
- MIDI-GPT: A Controllable Generative Model for Computer-Assisted Multitrack Music Composition (AAAI 2025 · similarity 0.88)
- MID-FiLD: MIDI Dataset for Fine-Level Dynamics (AAAI 2024 · similarity 0.87)
- Toward Multimodal Sheet Music Recognition and Understanding (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.