AI Paper Atlas

Research map / Probabilistic and scientific modeling / Molecules and proteins

Predictive Querying for Autoregressive Neural Sequence Models

Alex James Boyd, Sam Showalter, Stephan Mandt, Padhraic Smyth

NeurIPS 2022 · Probabilistic and scientific modeling › Molecules and proteins · approach: Neural point-process models · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use importance sampling instead of beam searchGPT-2 predictive hitting-time queries · WikiText, K=4, budget O(10^3)median RAE: 82\% -> 13\%Improved
use hybrid beam-plus-sampling estimator instead of pure importance samplingLSTM sequence models · Reviews, Mobile Apps, MOOCs, Shakespeare, K=3..11median RAEImproved
use importance sampling or hybrid instead of beam search as K growsLSTM sequence models · 3 of 4 datasets, K=3..11median RAEImproved
raise model temperature T (higher entropy)Beam search · Mobile Apps, tau(.)=4median RAE: error converges to 1Hurt
raise model temperature T (higher entropy)Importance sampling · Mobile Apps, tau(.)=4median RAE: error goes to 0Improved
use proposal-distribution importance sampling instead of naive Monte CarloK-step marginal estimation · all datasets, 4<=K<=15relative efficiency (variance ratio)Improved
increase query horizon KImportance sampling proposal · all datasets, 4<=K<=15relative efficiencyHurt
increase query horizon Kall estimators · four LSTM datasetsmedian RAEHurt
use coverage-based beam searchbeam search · query estimationcomputation costHurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy