Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning
Transcoda: End-to-End Zero-Shot Optical Music Recognition via Data-Centric Synthetic Training
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Transcoda trains a compact 59M-parameter ConvNeXt-V2 encoder-decoder on a large synthetic, target-normalized **kern corpus with grammar-constrained decoding, outperforming SMT++ and Legato on synthetic and zero-shot real Polish scans.
Problem
End-to-end optical music recognition is bottlenecked by the lack of large annotated real scan datasets, and the non-uniqueness of Humdrum **kern encodings creates a one-to-many mapping that makes learning and decoding harder.
Approach
Deterministically normalize **kern targets into a unique normal form via a 21-stage pipeline, pair this with a richer synthetic rendering and augmentation pipeline and concatenated multi-system pages, and add optional grammar-based constrained decoding for syntactic validity.
Key results
- On the synthetic Verovio benchmark, Transcoda with beam search reaches 18.46% OMR-NED versus 43.91% for Legato and 92.23% for SMT++.
- On real Polish scans, Transcoda reaches 63.97% OMR-NED versus 80.16% for SMT++ and 86.73% for Legato.
- Removing target normalization raises synthetic OMR-NED from 18.71% to 82.51%.
Surprising findings
- Removing target normalization causes catastrophic sequence collapse, raising OMR-NED from 18.71% to 82.51%.
- Constrained decoding yields negligible change in raw metrics while guaranteeing formal validity.
Limitations
- Structural decoding failures on real scans: hallucinated voice splits and merges cause drift in line widths.
- Rare (~1.18%) runaway generation loops producing valid but hallucinated repeated patterns, not caught by grammar constraints.
- Performance drops on dense, degraded, or handwritten real scans (semantic visual gap).
Experimental setup
Datasets: PDMX Grandstaff MuseTrainer OpenScore Lieder OpenScore Quartets Polish Scores IMSLP
Benchmarks: Verovio synthetic test split Polish historical scans (102 scores)
Metrics: OMR-NED TEDn CER
Baselines: SMT++ Legato
Builds on: SMT++ ConvNeXt-V2 FCMAE xgrammar Verovio
Closest papers by meaning
- Toward Multimodal Sheet Music Recognition and Understanding (NeurIPS 2026 · similarity 0.92)
- Rubato: Transcribing Piano Music with Timestamps (NeurIPS 2026 · similarity 0.86)
- Aria-MIDI: A Dataset of Piano MIDI Files for Symbolic Music Modeling (ICLR 2025 · similarity 0.86)
- UniMuMo: Unified Text, Music, and Motion Generation (AAAI 2025 · similarity 0.84)
- V2Meow: Meowing to the Visual Beat via Video-to-Music Generation (AAAI 2024 · similarity 0.84)
- Zero-Shot Unsupervised and Text-Based Audio Editing Using DDPM Inversion (ICML 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.