AI Paper Atlas

Research map / Architectures and efficiency / Attention and transformers

Generating Sentences Using a Dynamic Canvas

Harshil Shah, Bowen Zheng, David Barber

AAAI 2018 · Architectures and efficiency › Attention and transformers · approach: Language model pretraining and induction · 2 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace the sequential Gen-RNN generator with AUTR's attention-based dynamic canvassentence generation · Book Corpus testELBO: -52.3 -> -50.7Improved
replace the sequential Gen-RNN generator with AUTR's attention-based dynamic canvassentence generation · Book Corpus testPPL: 41.9 -> 37.4Improved
replace the sequential Gen-RNN generator with AUTR's attention-based dynamic canvassentence generation · Book Corpus testKL: 7.1 -> 8.0Improved
add 30% word dropout (ablation)AUTR, T=30 · Book Corpus testELBO: -50.7 -> -51.6Hurt
add 30% word dropout (ablation)AUTR, T=40 · Book Corpus testELBO: -50.7 -> -51.5Hurt
use T=40 instead of T=30 (ablation)AUTR without dropout · Book Corpus testPPL: 37.4 -> 37.4No clear effect
use T=40 instead of T=30 (ablation)AUTR with 30% dropout · Book Corpus testELBO: -51.6 -> -51.5No clear effect
add 30% word dropout (ablation)AUTR · Book Corpus testKL: 8.0 -> 14.0Improved
use AUTR instead of Gen-RNN for iterative missing-word imputationmissing-word imputation · Book Corpus test, 30% words missingmissing-word imputation accuracy: 31.9% -> 34.1%Improved
use modified attention that favors unwritten slots and caps cumulative attention (ablation)AUTR canvas updates · Book Corpusmodel performanceImproved
use dynamic attention instead of a fixed canvas update mechanism (ablation)AUTR canvas updates · Book Corpususe of recurrent computation stepsImproved
condition each generated word on preceding words rather than generate words independently (ablation)AUTR text decoder · Book Corpus sentence generationlocal word consistencyImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy