Research map / Probabilistic and scientific modeling / Variational and generative sampling
Deep State Space Models for Unconditional Word Generation
Florian Schmidt, Thomas Hofmann
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add a second tril transition flow (ablation) | Deep State Space Model · BooksCorpus standard split | H[P_test, P̂]: 11.99\ensuremath{\oldpm}.11 -> 11.86\ensuremath{\oldpm}.13 | Improved |
| add importance weighting with K=10 (ablation) | 2×tril model · BooksCorpus standard split | H[P_test, P̂]: 11.86\ensuremath{\oldpm}.13 -> 11.38\ensuremath{\oldpm}.06 | Improved |
| increase importance samples from K=5 to K=10 (ablation) | 2×tril model · BooksCorpus standard split | H[P_test, P̂]: 11.46\ensuremath{\oldpm}.06 -> 11.38\ensuremath{\oldpm}.06 | No clear effect |
| replace the standard inference factorization with bidirectional conditioning (ablation) | 2×tril, K=10 model · BooksCorpus standard split | H[P_test, P̂]: 11.38\ensuremath{\oldpm}.06 -> 11.39\ensuremath{\oldpm}.10 | No clear effect |
| replace tril flows with Real NVP flows (ablation) | Deep State Space Model · BooksCorpus standard split | H[P_test, P̂]: 11.38\ensuremath{\oldpm}.06 -> 11.81 | Hurt |
| increase state size to d=16 (ablation) | 2×tril, K=10 model · BooksCorpus standard split | H[P_test, P̂]: 11.38\ensuremath{\oldpm}.06 -> 11.43 | No clear effect |
| increase state size to d=32 (ablation) | 2×tril, K=10 model · BooksCorpus standard split | H[P_test, P̂]: 11.38\ensuremath{\oldpm}.06 -> 11.13 | Improved |
| use the non-autoregressive state-space model instead of the teacher-forced GRU baseline | BooksCorpus generation · BooksCorpus standard split | H[P_test, P̂]: 12.97 -> 11.38\ensuremath{\oldpm}.06 | Improved |
| use the non-autoregressive state-space model instead of the teacher-forced GRU baseline | BooksCorpus generation · BooksCorpus alternative vocabulary split | H[P_test, P̂]: 13.67 -> 12.27 | Improved |
Closest papers by meaning
- Compressed and Smooth Latent Space for Text Diffusion Modeling (NeurIPS 2025 · similarity 0.88)
- Reflection-Window Decoding: Text Generation with Selective Refinement (ICML 2025 · similarity 0.87)
- Flexible Language Modeling in Continuous Space with Transformer-based Autoregressive Flows (NeurIPS 2025 · similarity 0.86)
- Tractable Transformers for Flexible Conditional Generation (ICML 2025 · similarity 0.86)
- Guaranteed Generation from Large Language Models (ICLR 2025 · similarity 0.85)
- The Hyperfitting Phenomenon: Sharpening and Stabilizing LLMs for Open-Ended Text Generation (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
