Research map / Image generation / Autoregressive and latent generation
TechSinger: Technique Controllable Multilingual Singing Voice Synthesis via Flow Matching
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TechSinger, a flow-matching multilingual singing voice synthesis system with phoneme-level vocal technique control, an automatic technique detector for data annotation, and a prompt-based technique predictor.
Problem
Existing singing voice synthesis offers little precise control over vocal techniques (e.g., intensity, mixed voice, falsetto, bubble, breathy), and technique-annotated data is scarce.
Approach
Uses rectified flow matching for both F0 prediction (FMPP) and mel-spectrogram refinement (CFG flow-matching postnet) conditioned on phoneme-level technique labels, auto-annotates open datasets with a trained technique detector, and maps natural-language prompts to technique sequences via a FLAN-T5-based predictor.
Key results
- TechSinger achieves MOS-Q 3.89 and MOS-C 4.10, above StyleSinger (3.69 / 3.93), DiffSinger, and VISinger2.
- TechSinger obtains the lowest FFE (0.245) and MCD (3.823) among the compared systems.
- Prompt-based technique prediction (MOS-C 4.04) is close to ground-truth technique labels (4.10) and far above random techniques (3.76).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TechSinger instead of the strongest listed SVS baseline | Technique-controllable singing voice synthesis · Technique-controllable singing voice synthesis | MOS-Q: 3.69 $\pm$ 0.09 -> 3.89 $\pm$ 0.07 | Helped |
| Use TechSinger instead of the strongest listed SVS baseline | Technique-controllable singing voice synthesis · Technique-controllable singing voice synthesis | MOS-C: 3.93 $\pm$ 0.08 -> 4.10 $\pm$ 0.08 | Helped |
| Use TechSinger instead of the strongest listed SVS baseline | Technique-controllable singing voice synthesis · Technique-controllable singing voice synthesis | FFE: 0.328 -> 0.245 | Helped |
| Use TechSinger instead of the strongest listed SVS baseline | Technique-controllable singing voice synthesis · Technique-controllable singing voice synthesis | MCD: 3.981 -> 3.823 | Helped |
| Add Squeezeformer instead of using convolution in the technique detector (ablation) | TechSinger technique detector · Technique detector | F1: 0.742 -> 0.770 | Helped |
| Use multi-head weight prediction instead of averaging (ablation) | TechSinger technique detector · Technique detector | F1: 0.763 -> 0.770 | Helped |
| Add the flow-matching pitch predictor (ablation) | TechSinger · Technique-controllable singing voice synthesis | CMOSQ: -0.25 -> 0.00 | Helped |
| Add the flow-matching pitch predictor (ablation) | TechSinger · Technique-controllable singing voice synthesis | FFE: 0.2537 -> 0.2448 | Helped |
| Add the postnet (ablation) | TechSinger · Technique-controllable singing voice synthesis | CMOSQ: -0.33 -> 0.00 | Helped |
| Add classifier-free guidance (ablation) | TechSinger postnet · Technique-controllable singing voice synthesis | CMOSQ: -0.10 -> 0.00 | Helped |
| Add classifier-free guidance (ablation) | TechSinger postnet · Technique-controllable singing voice synthesis | FFE: 0.2453 -> 0.2448 | Helped |
Limitations
- Technique detector is weak on bubble and vibrato (e.g., vibrato F1 0.374), so auto-labeled data is noisy.
- Ethical concerns about displacing singers and copyright; code and models are said to be restricted.
- Baselines were augmented with a technique embedding layer, since existing SVS models lack technique control.
Experimental setup
Datasets: GTSinger M4Singer in-house 30-hour Chinese technique dataset
Benchmarks: GTSinger test segments (804 segments)
Metrics: MOS-Q MOS-C FFE MCD precision recall F1 accuracy
Baselines: DiffSinger VISinger2 StyleSinger
Builds on: rectified flow matching DiffSinger FastSpeech2 ROSVOT Squeezeformer
Closest papers by meaning
- GTSinger: A Global Multi-Technique Singing Corpus with Realistic Music Scores for All Singing Tasks (NeurIPS 2024 · similarity 0.89)
- SongGen: A Single Stage Auto-regressive Transformer for Text-to-Song Generation (ICML 2025 · similarity 0.87)
- NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizers (ICLR 2024 · similarity 0.86)
- VocalCoachBench: Benchmarking Audio-Language Models on Expert Feedback for Singing (NeurIPS 2026 · similarity 0.86)
- CSSinger: End-to-End Chunkwise Streaming Singing Voice Synthesis System Based on Conditional Variational Autoencoder (AAAI 2025 · similarity 0.85)
- SongCreator: Lyrics-based Universal Song Generation (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.