Research map / Image generation / Autoregressive and latent generation
ReGen: Hierarchical Multi-Prompt Representation Generation for Efficient Waveform Diffusion Models
Sang-Hoon Lee, Ha-Yeong Choi
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add REPA with Mel-spectrogram alignment (ablation) | waveform diffusion codec · LibriTTS reconstruction, waveform diffusion codec, 0.5M steps | WER: 136.57 -> 15.10 | Improved |
| add REPA with SSL representation only (ablation) | waveform diffusion codec · LibriTTS reconstruction, waveform diffusion codec, 0.5M steps | WER: 136.57 -> 130.64 | No clear effect |
| use hierarchical REPA (SSL, Mel) instead of Mel-only REPA (ablation) | waveform diffusion codec · LibriTTS reconstruction, waveform diffusion codec, 0.5M steps | SPK-SIM: 0.32 -> 0.38 | Improved |
| replace REPA with representation generation (ReGen-H, SSL and Mel) (ablation) | waveform diffusion codec · LibriTTS reconstruction, waveform diffusion codec, 0.5M steps | WER: 16.53 -> 11.99 | Improved |
| replace REPA with representation generation (ReGen-H, SSL and Mel) (ablation) | waveform diffusion codec · LibriTTS reconstruction, waveform diffusion codec, 0.5M steps | SPK-SIM: 0.38 -> 0.54 | Improved |
| use unified generation at same layer instead of hierarchical (ablation) | ReGen · LibriTTS reconstruction, waveform diffusion codec, 0.5M steps | WER: 11.99 -> 89.94 | Hurt |
| lower representation generation weight to 0.01 in unified ReGen (ablation) | ReGen-U · LibriTTS reconstruction, waveform diffusion codec, 0.5M steps | WER: 89.94 -> 19.31 | Improved |
| add generalized flow matching (repulsive term) (ablation) | ReGen-H · LibriTTS reconstruction, waveform diffusion codec, 0.5M steps | SPK-SIM: 0.54 -> 0.59 | Improved |
| add generalized flow matching (repulsive term) (ablation) | ReGen-H · LibriTTS reconstruction, waveform diffusion codec, 0.5M steps | WER: 11.99 -> 12.59 | No clear effect |
| add adversarial post-training (ablation) | ReGenVAE · LibriTTS reconstruction | WER: 3.63 -> 2.88 | Improved |
| add auxiliary decoder (CosyVoice2 token distillation on FSQ) (ablation) | ReGenTokenizer · LibriTTS reconstruction, waveform diffusion codec, 0.5M steps | WER: 12.59 -> 6.01 | Improved |
| add GFM objective to LDM-based TTS (ablation) | ReGenVoice · Seed-en zero-shot TTS | WER: 1.60 -> 1.46 | Improved |
Closest papers by meaning
- MDSGen: Fast and Efficient Masked Diffusion Temporal-Aware Transformers for Open-Domain Sound Generation (ICLR 2025 · similarity 0.88)
- Fast Timing-Conditioned Latent Audio Diffusion (ICML 2024 · similarity 0.88)
- A Versatile Diffusion Transformer with Mixture of Noise Levels for Audiovisual Generation (NeurIPS 2024 · similarity 0.87)
- DiffAR: Denoising Diffusion Autoregressive Model for Raw Speech Waveform Generation (ICLR 2024 · similarity 0.87)
- Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think (ICLR 2025 · similarity 0.87)
- DiTTo-TTS: Diffusion Transformers for Scalable Text-to-Speech without Domain-Specific Factors (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
