Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning
StyleSinger: Style Transfer for Out-of-Domain Singing Voice Synthesis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes StyleSinger, a zero-shot style-transfer singing voice synthesis model that uses a residual-quantization style adaptor and uncertainty-modeling layer norm to transfer out-of-domain singing styles from reference audio.
Problem
Singing voice synthesis with zero-shot transfer of out-of-domain styles (timbre, emotion, pronunciation, articulation skills) from reference singing samples, where existing SVS models degrade on unseen styles.
Approach
A Residual Style Adaptor uses residual quantization over reference mel-spectrograms and f0 to capture fine-grained style (pronunciation, articulation), and Uncertainty Modeling Layer Normalization perturbs style statistics during training to improve generalization to unseen styles, combined with a pitch diffusion predictor and diffusion mel decoder.
Key results
- StyleSinger achieves MOS 3.90 and SMOS 4.03, the highest among baselines (MS RMSSinger MOS 3.84, SMOS 3.90), with Cos 0.90 and FFE 0.27.
- AXY preference tests show StyleSinger preferred over Styler, GenerSpeech, YourTTS and MS RMSSinger in both parallel and non-parallel transfer.
- Ablation CMOS drops for removing UMLN (-0.28), RSA (CSMOS -0.29), pitch diffusion predictor (-0.40) and diffusion decoder (-0.52).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use StyleSinger instead of MS RMSSinger | OOD parallel style transfer · OOD parallel style transfer | MOS: 3.84 ± 0.08 -> 3.90 ± 0.05 | Helped |
| use StyleSinger instead of MS RMSSinger | OOD parallel style transfer · OOD parallel style transfer | SMOS: 3.90 ± 0.11 -> 4.03 ± 0.07 | Helped |
| use StyleSinger instead of MS RMSSinger | OOD parallel style transfer · OOD parallel style transfer | Cos: 0.88 -> 0.90 | Helped |
| use StyleSinger instead of MS RMSSinger | OOD parallel style transfer · OOD parallel style transfer | FFE: 0.29 -> 0.27 | Helped |
| add UMLN (ablation) | StyleSinger · OOD parallel style transfer | CMOS: -0.28 -> 0.00 | Helped |
| add UMLN (ablation) | StyleSinger · OOD parallel style transfer | CSMOS: -0.25 -> 0.00 | Helped |
| add RSA (ablation) | StyleSinger · OOD parallel style transfer | CMOS: -0.12 -> 0.00 | Helped |
| add RSA (ablation) | StyleSinger · OOD parallel style transfer | CSMOS: -0.29 -> 0.00 | Helped |
| use pitch diffusion predictor instead of the FastSpeech2 simple pitch predictor (ablation) | StyleSinger · OOD parallel style transfer | CMOS: -0.40 -> 0.00 | Helped |
| use pitch diffusion predictor instead of the FastSpeech2 simple pitch predictor (ablation) | StyleSinger · OOD parallel style transfer | CSMOS: -0.23 -> 0.00 | Helped |
| use diffusion decoder instead of a transformer decoder (ablation) | StyleSinger · OOD parallel style transfer | CMOS: -0.52 -> 0.00 | Helped |
| use diffusion decoder instead of a transformer decoder (ablation) | StyleSinger · OOD parallel style transfer | CSMOS: -0.19 -> 0.00 | Helped |
Limitations
- Own singing corpus is not publicly released, and the ethics statement says code and models will be restricted.
- Evaluation limited to Chinese songs and a small OOD test set of 20 sentences.
- Relies on subjective listening tests with limited sample size for style judgments.
Experimental setup
Datasets: M4Singer in-house Chinese singing corpus
Benchmarks: OOD singing style-transfer test set (20 sentences, unseen styles)
Metrics: MOS SMOS Cos (speaker cosine similarity) FFE AXY preference CMOS CSMOS
Baselines: Styler GenerSpeech YourTTS MS RMSSinger Reference (vocoder)
Builds on: FastSpeech2 DiffSinger ProDiff wav2vec 2.0 BigVGAN Conditional layer normalization
Closest papers by meaning
- NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizers (ICLR 2024 · similarity 0.87)
- CSSinger: End-to-End Chunkwise Streaming Singing Voice Synthesis System Based on Conditional Variational Autoencoder (AAAI 2025 · similarity 0.87)
- GTSinger: A Global Multi-Technique Singing Corpus with Realistic Music Scores for All Singing Tasks (NeurIPS 2024 · similarity 0.86)
- MM-TTS: Multi-Modal Prompt Based Style Transfer for Expressive Text-to-Speech Synthesis (AAAI 2024 · similarity 0.86)
- DDDM-VC: Decoupled Denoising Diffusion Models with Disentangled Representation and Prior Mixup for Verified Robust Voice Conversion (AAAI 2024 · similarity 0.86)
- AnyEdit: A Unified Framework for Speech and Singing Voice Editing with Real-World Environmental Consistency (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.