Research map / Large language models / Language, safety and interpretability
Flow of Spans: Generalizing Language Models to Dynamic Span-Vocabulary via GFlowNets
Bo Xue, Yunchong Song, Fanghao Shao, Xuekai Zhu, Lin Chen, Luoyi Fu, Xinbing Wang, Zhouhan Lin
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| model the state space as a DAG with phrases instead of a tree (ablation) | FoSS · WikiText-103 in-domain, nucleus | MAUVE: 29.61 -> 31.65 | Improved |
| model the state space as a DAG with phrases instead of a tree (ablation) | FoSS · WikiText-103 in-domain, nucleus | Diversity: 65.72 -> 92.48 | Improved |
| add preference model reward (ablation) | FoSS · WikiText-103 in-domain, nucleus | MAUVE: 28.25 -> 31.65 | Improved |
| add language model reward (ablation) | FoSS · WikiText-103 in-domain, nucleus | MAUVE: 29.09 -> 31.65 | Improved |
| add language model reward on top of preference model (ablation) | FoSS · WikiText-103 in-domain, nucleus | Diversity: 92.77 -> 92.48 | No clear effect |
| add language model reward (ablation) | FoSS · Law-MT out-of-domain, nucleus | MAUVE: 29.79 -> 32.17 | Improved |
| use DAG span state space with GFlowNets instead of tree GFlowNets-FT | FoSS · WikiText-103 in-domain, nucleus | MAUVE: 29.61 -> 31.65 | Improved |
| use FoSS instead of CoG | FoSS · WikiText-103 in-domain, nucleus | MAUVE: 26.14 -> 31.65 | Improved |
| use FoSS instead of fine-tuned Transformer | FoSS · Law-MT out-of-domain, nucleus | MAUVE: 26.85 -> 32.17 | Improved |
| use larger En-Wiki data store without retraining | FoSS · WikiText-103 test, nucleus | MAUVE: 31.65 -> 33.79 | Improved |
| use FoSS instead of fine-tuned Transformer | FoSS · OpenBookQA | accuracy (%): 22.71 -> 26.20 | Improved |
| use FoSS instead of CoG | FoSS · MedMCQA | accuracy (%): 27.55 -> 27.44 | Hurt |
Closest papers by meaning
- From Bytes to Ideas: Language Modeling with Autoregressive U-Nets (NeurIPS 2025 · similarity 0.87)
- Chunk-Distilled Language Modeling (ICLR 2025 · similarity 0.87)
- Large Language Models Explore by Latent Distilling (ICML 2026 · similarity 0.86)
- Retrieval is Accurate Generation (ICLR 2024 · similarity 0.86)
- Flexible Language Modeling in Continuous Space with Transformer-based Autoregressive Flows (NeurIPS 2025 · similarity 0.86)
- Flow Map Language Models: One-step Language Modeling via Continuous Denoising (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
