Research map / Image generation / Autoregressive and latent generation
Neural Audio Synthesis of Musical Notes with WaveNet Autoencoders
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use a WaveNet autoencoder instead of the spectral autoencoder | NSynth note reconstruction · NSynth held-out reconstructions | pitch classification accuracy: 46.9% -> 79.6% | Helped |
| use a WaveNet autoencoder instead of the spectral autoencoder | NSynth note reconstruction · NSynth held-out reconstructions | quality classification accuracy: 85.2% -> 88.9% | Helped |
| add pitch conditioning during training (ablation) | WaveNet autoencoder, Z size 1984 · NSynth held-out embeddings | pitch classification accuracy: 58.1 -> 40.5 | Hurt |
| add pitch conditioning during training (ablation) | spectral autoencoder, Z size 1984 · NSynth held-out embeddings | pitch classification accuracy: 63.8 -> 55.2 | Hurt |
| add pitch conditioning during training (ablation) | spectral autoencoder, Z size 1024 · NSynth held-out embeddings | pitch classification accuracy: 57.4 -> 42.1 | Hurt |
| add pitch conditioning during training (ablation) | spectral autoencoder, Z size 512 · NSynth held-out embeddings | pitch classification accuracy: 63.2 -> 21.8 | Hurt |
| add pitch conditioning during training (ablation) | spectral autoencoder, Z size 256 · NSynth held-out embeddings | pitch classification accuracy: 57.7 -> 21.0 | Hurt |
| add pitch conditioning during training (ablation) | spectral autoencoder, Z size 128 · NSynth held-out embeddings | pitch classification accuracy: 58.2 -> 21.2 | Hurt |
| add pitch conditioning during training (ablation) | spectral autoencoder, Z size 64 · NSynth held-out embeddings | pitch classification accuracy: 59.8 -> 15.2 | Hurt |
| reduce embedding size while adding pitch conditioning (ablation) | spectral autoencoder · NSynth held-out embeddings | relative change in pitch classification accuracy: -13.5 -> -74.6 | Mixed |
| train the spectral autoencoder on peak-normalized log magnitude spectra and reconstruct phase with Griffin-Lim (ablation) | spectral autoencoder baseline · NSynth note reconstruction | perceptual sample quality | Helped |
Closest papers by meaning
- SING: Symbol-to-Instrument Neural Generator (NeurIPS 2018 · similarity 0.91)
- The challenge of realistic music generation: modelling raw audio at scale (NeurIPS 2018 · similarity 0.90)
- Contrastive Learning from Synthetic Audio Doppelgängers (ICLR 2025 · similarity 0.87)
- DFlow: A Generative Model Combining Denoising AutoEncoder and Normalizing Flow for High Fidelity Waveform Generation (ICML 2024 · similarity 0.87)
- MeLFusion: Synthesizing Music from Image and Language Cues using Diffusion Models (CVPR 2024 · similarity 0.87)
- Fast Timing-Conditioned Latent Audio Diffusion (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.