Research map / Architectures and efficiency / RNNs and normalization
From Isolated feature to Orbits:\\Discovering Music Concepts via Multi-SAE Alignment
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes multi-SAE alignment with shared top-K support over pitch-shifted views of frozen music foundation model features to recover pitch-transposition orbits of chord, key, and melody features.
Problem
Interpreting music foundation models beyond isolated SAE features: many musical concepts (chords, keys, melodies) are structured relations under pitch transposition, but existing probing and SAE analyses identify only individual feature-concept correspondences.
Approach
Train one Top-K SAE per pitch-shifted view of the same audio with a shared top-K support mask so same-index features align across views; then derive a successor map via decoder cosine similarity to recover ring/chain orbits of features that encode transposition-equivariant concepts.
Key results
- On chord recognition, MuQ o-SAE with oracle boundaries reaches 85.29 Root WCR on POP909, close to the supervised probe (84.17) and LVCR (84.25), but below LVCR on RWC (86.21 vs 87.96).
- Key detection on GTZAN-Key (song-level accuracy) with MuQ o-SAE gives 51.57 vs 48.80 for s-SAE and 48.80 for probe, though S-KEY reaches 65.54.
- A 12-feature subdominant-centered ring is found in MuQ but not in MusicFM, while major and minor chord rings appear in both models.
Surprising findings
- Linear probes detect weak tonic and dominant signals that SAE does not surface as salient features; key information can be reconstructed from chord activations, suggesting linear recoverability does not imply explicit concept organization.
- Tonic and dominant rings are absent from recovered orbits despite being linearly decodable, while the subdominant ring is recovered.
Limitations
- Orbits are characterized only by pitch transposition; pitch-invariant concepts such as rhythm cannot be organized without extra inductive biases.
- Frame-level focus; how frame-level concepts are aggregated over time is unexplored.
- Training on synthesized, vocal-free Slakh2100 yields mostly Western music concepts and a domain gap to real recordings.
Experimental setup
Datasets: Slakh2100 POP909 RWC-POP-100 FMAKv2 GTZAN-Key GiantSteps
Benchmarks: POP909 RWC-POP-100 Slakh2100 test FMAKv2 GTZAN-Key GiantSteps
Metrics: Root WCR MajMin WCR key accuracy (song/segment) MIREX weighted accuracy
Baselines: Linear probe standard SAE (s-SAE) S-KEY LVCR
Builds on: Top-K SAE Universal Sparse Autoencoders S-KEY LVCR
Closest papers by meaning
- N-gram Unsupervised Compoundation and Feature Injection for Better Symbolic Music Understanding (AAAI 2024 · similarity 0.85)
- Whole-Song Hierarchical Generation of Symbolic Music Using Cascaded Diffusion Models (ICLR 2024 · similarity 0.85)
- MusER: Musical Element-Based Regularization for Generating Symbolic Music with Emotion (AAAI 2024 · similarity 0.84)
- Toward Multimodal Sheet Music Recognition and Understanding (NeurIPS 2026 · similarity 0.83)
- Unifying Symbolic Music Arrangement: Track-Aware Reconstruction and Structured Tokenization (NeurIPS 2025 · similarity 0.83)
- LLark: A Multimodal Instruction-Following Language Model for Music (ICML 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.