AI Paper Atlas

Research map / Label-efficient and robust learning / Self-supervised and contrastive

HierSpeech: Bridging the Gap between Text and Speech by Hierarchical Variational Inference using Self-supervised Representations for Speech Synthesis

Sang-Hoon Lee, Seung-Bin Kim, Ji-Hyun Lee, Eunwoo Song, Min-Jae Hwang, Seong-Whan Lee

NeurIPS 2022 · Label-efficient and robust learning › Self-supervised and contrastive · approach: Self-supervised speech and brain decoding · 2 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use linear spectrogram instead of 1st layer of XLS-R as speaker encoder input (ablation)HierSpeech · speaker encoder inputEER: 4.62 -> 2.77Improved
use 12th layer of XLS-R instead of 1st layer as speaker encoder input (ablation)HierSpeech · speaker encoder inputEER: 4.62 -> 10.23Hurt
use 12th layer of XLS-R for the linguistic encoder (zl) rather than 1st layer (ablation)HierSpeech · frame-level speaker classificationspeaker accuracy of zl: 15.09% -> 8.95%Improved
increase VITS flow blocks from 4 to 8 (ablation)VITS · VCTK ablationPER: 12.24 -> 13.42Hurt
add phoneme predictor (PP) to VITS posterior encoder (ablation)VITS · VCTK ablationPER: 12.24 -> 7.60Improved
add phoneme predictor (PP) to VITS posterior encoder (ablation)VITS · VCTK ablationWER: 30.62 -> 22.98Improved
add acoustic encoder (AE) between linguistic representation and waveform decoder (ablation)HierSpeech · VCTK ablationPER: 8.00 -> 6.25Improved
add both AE and PP to the model (ablation)HierSpeech · VCTK ablationPER: 16.70 -> 6.25Improved
replace VITS with HierSpeech hierarchical variational inference (ablation)VITS · VCTK TTSWER: 25.54 -> 19.55Improved
replace VITS with HierSpeech (ablation)VITS · LibriTTS speaker transferMCD: 4.37 -> 4.42Hurt
replace VITS with HierSpeech (ablation)VITS · LibriTTS speaker transferPER: 13.62 -> 7.47Improved
fine-tune without transcripts (HierSpeech-U) instead of with transcripts (ablation)HierSpeech · few-shot speaker adaptation, VCTKWER: 16.95 -> 15.85Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy