AI Paper Atlas

Research map / Large language models / Language, safety and interpretability

Spell Once, Summon Anywhere: A Two-Level Open-Vocabulary Language Model

Sebastian J. Mielke, Jason Eisner

AAAI 2019 · Large language models › Language, safety and interpretability · approach: Controlled text generation

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Combine type-level spelling training and UNK-token spelling training in one model (ablation)Spell Once · WikiText-2bits per character: 1.455 vs 1.462Improved
Add spelling training on UNK tokens (ablation)Spell Once · WikiText-2bits per character: 1.455 vs 1.494Improved
Share the spelling model between in-vocabulary types and UNK tokens (ablation)Spell Once · WikiText-2bits per character: 1.455 vs 1.469No clear effect
Condition the spelling RNN on the word embedding (ablation)Spell Once · WikiText-2bits per character: 1.468 -> 1.455Improved
Use an RNN spelling model instead of a character unigram model (ablation)Spell Once · WikiText-2bits per character: 1.548 -> 1.455Improved
Use Spell Once instead of pure BPELanguage modeling · WikiText-2bits per character: 1.468 -> 1.455Improved
Use Spell Once instead of pure BPEMWC English · tokenize testbits per character: 1.386 vs 1.387No clear effect
Use Spell Once instead of pure BPEMWC French · tokenize testbits per character: 1.317 vs 1.319No clear effect
Use Spell Once instead of pure BPEMWC German · tokenize testbits per character: 1.414 -> 1.465Hurt
Use Spell Once instead of pure BPEMWC Spanish · tokenize testbits per character: 1.362 vs 1.363No clear effect
Use Spell Once instead of pure BPEMWC Czech · tokenize testbits per character: 1.856 -> 1.928Hurt
Use Spell Once instead of pure BPEMWC Finnish · tokenize testbits per character: 1.652 -> 1.751Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy