Research map / Image generation / Autoregressive and latent generation
Topic-VQ-VAE: Leveraging Latent Codebooks for Flexible Topic-Guided Document Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TVQ-VAE, a generative topic model that treats VQ-VAE codebooks (multi-hot, expanded via K-nearest embeddings) as conceptual words and generates either BoW documents or autoregressive image codebook sequences conditioned on topics.
Problem
Topic models are limited to traditional bag-of-words output and do not exploit discrete pre-trained (PLM or visual) embeddings flexibly; there is a need for topic-guided generation beyond BoW form, e.g., autoregressive image generation.
Approach
Interpret VQ-VAE codebooks (multi-hot, expanded from K-nearest embeddings) and their embeddings as conceptual words, and place a VQ-VAE-based output distribution (BoW softmax or PixelCNN/Transformer over codebook sequences) inside a ProdLDA-style variational topic model so topics can drive flexible document generation.
Key results
- On NYT, TVQ-VAE reaches Km-NMI 0.184 and Km-Purity 0.510, the best among compared topic models (ETM, LDA, ProdLDA, BERTopic, TopClus).
- On 20NG, ETM has the best Km-NMI (0.331), while TVQ-VAE gets 0.210 Km-NMI and 0.242 Km-Purity, competitive with LDA and ProdLDA.
- On CIFAR-10, NLL decreases from 3.29 (unsupervised) to 3.20 with 100 topics, while on CelebA the unsupervised baseline has lower NLL (3.10 vs 3.13 at 100 topics).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TVQ-VAE topic conditioning instead of unsupervised generation | VQ-VAE + PixelCNN · CelebA | NLL: 3.10 -> 3.13 | Hurt |
| Use TVQ-VAE with 100 topics instead of unsupervised generation | VQ-VAE + PixelCNN · CIFAR-10 | NLL: 3.29 -> 3.20 | Helped |
| Increase codebook count and expansion value (ablation) | TVQ-VAE · 20NG | TQ | No clear effect |
| Increase codebook count and expansion value (ablation) | TVQ-VAE · NYT | TQ | Helped |
Surprising findings
- On CelebA, the unsupervised baseline yields lower NLL than the topic-conditioned TVQ-VAE (P), contrary to the CIFAR-10 trend.
- On 20NG, varying codebook size and expansion value shows minimal performance differences, suggesting larger settings do not necessarily help on small vocabularies.
Limitations
- Image generation experiments are compared only against the authors' own VQ-VAE+PixelCNN baseline, since the general topic-guided generation task is new.
- On 20NG, performance is largely insensitive to codebook size and expansion value, and ETM/LDA match or beat TVQ-VAE on some metrics.
- The VQ-VAE encoder is kept fixed rather than trained jointly.
Experimental setup
Datasets: 20 Newsgroups New York Times annotated corpus CIFAR-10 CelebA FacesHQ
Benchmarks: 20 Newsgroups NYT CIFAR-10 CelebA
Metrics: NPMI topic diversity topic quality (TC x TD) Km-NMI Km-Purity NLL
Baselines: LDA ProdLDA ETM BERTopic TopClus VQ-VAE with PixelCNN prior
Builds on: VQ-VAE ProdLDA ETM PixelCNN
Closest papers by meaning
- Alleviating "Posterior Collapse'' in Deep Topic Models via Policy Gradient (NeurIPS 2022 · similarity 0.86)
- Sparse Autoencoders are Topic Models (ICML 2026 · similarity 0.86)
- Bayesian Deep Embedding Topic Meta-Learner (ICML 2022 · similarity 0.86)
- Bayesian Progressive Deep Topic Model with Knowledge Informed Textual Data Coarsening Process (ICML 2023 · similarity 0.86)
- Document Summarization with VHTM: Variational Hierarchical Topic-Aware Mechanism (AAAI 2020 · similarity 0.86)
- Dispersed Exponential Family Mixture VAEs for Interpretable Text Generation (ICML 2020 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.