AI Paper Atlas

Research map / Image generation / Autoregressive and latent generation

Latent Diffusion for Language Generation

Justin Lovelace, Varsha Kishore, Chao Wan, Eliot Seo Shekhtman, Kilian Q Weinberger

NeurIPS 2023 · Image generation › Autoregressive and latent generation · approach: Latent diffusion with learned tokenizers · 16 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
diffuse in compressed 32x64 latent space instead of full encoder features (ablation)LD4LG (BART-base) · ROCStories unconditionalMAUVE: .605 -> .716Improved
diffuse in compressed latent space instead of full encoder features (ablation)LD4LG (BART-base) · ROCStories unconditionalrelative training speedup: 1.0x -> 3.86xImproved
add self-conditioning (ablation)LD4LG (BART-base) · ROCStories unconditionalMAUVE: .480 -> .716Improved
add self-conditioning (ablation)LD4LG (BART-base) · ROCStories unconditionalperplexity: 79.3 -> 30.6Improved
add self-conditioning (ablation)LD4LG (BART-base) · ROCStories unconditionaldiversity: .427 -> .331Hurt
use LD4LG instead of Diffusion-LMDiffusion models · ROCStories unconditionalMAUVE: .043 -> .716Improved
use LD4LG instead of GPT-2-MediumLanguage models · AG News unconditionalmemorization: .829 -> .293Improved
use LD4LG (BART) instead of fine-tuned GPT-2-MediumLanguage models · ROCStories unconditionalMAUVE: .788 -> .716Hurt
use LD4LG instead of DiffuSeqSeq2seq diffusion · XSumROUGE-L: 14.1 -> 31.9Improved
increase sampling steps from 250 to 1000 (ablation)LD4LG (BART-base) · ROCStories unconditionalMAUVE: .716 -> .667Mixed
reduce latent dimensionality from 32x64 to 32x32 (ablation)BART autoencoder · ROCStories unconditionalreconstruction BLEU: 97.6 -> 92.4Hurt
use classifier-free guidance w=2.0 (ablation)LD4LG · XSum validationROUGE-LImproved

Earlier methods this paper compares with

Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.

Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy