AI Paper Atlas

Research map / Probabilistic and scientific modeling / Variational and generative sampling

Foundation Posteriors for Approximate Probabilistic Inference

Mike Wu, Noah Goodman

NeurIPS 2022 · Probabilistic and scientific modeling › Variational and generative sampling · approach: Exact inference for probabilistic programs · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
remove the MLM term (train inference head only) (ablation)MLI · six toy probabilistic programslog p(z|x): -3.740 -> -1.538 (Latent)Improved
remove the MLM term (ablation)MLI · Hierarchical toy programvariance of log IW: 5.162e9 -> 119.2Improved
add program augmentations to meta-training (ablation)MLI · held-out Rosenbrocklog p(z|x): -246.6 -> -8.160Improved
add program augmentations to meta-training (ablation)MLI · held-out Rosenbrockvariance of log IW: 355.9 -> 15.10Improved
add variational finetuning (1,000 SVI iterations) after MLI pretrainingMLI · held-out Rosenbrocklog p(z|x): -8.160 -> -5.727Improved
finetune from random initialization instead of MLI pretrained weights (ablation)finetuning · held-out Rosenbrocklog p(z|x): -5.727 -> -42.54Hurt
initialize transformer backbone with CodeBERT instead of RoBERTa (ablation)MLI · six toy programsvariance of log IW: 131.3 -> 69.47 (Multi-level)Improved
initialize backbone with CodeBERT instead of RoBERTa (ablation)MLI · six toy programslog p(z|x): -1.827 -> -2.041 (Rosenbrock); -3.268 -> -3.691 (Hierarchical); -3.359 -> -3.054 (Multi-level)Mixed
output scientific notation (a x 10^b) from the inference head (ablation)MLI inference head · toy programstraining stability and average performanceHurt
finetune the model on all 30 responses instead of zero-shot with k=1 observationsplated MLI · plated IRTlog p(z|x): 0.5174 -> 0.8743Improved
finetune instead of product of experts at k=5plated MLI · plated IRTlog p(z|x): 1.3201 -> 1.7331Improved
increase minibatch size k from 1 to 5zero-shot plated MLI · plated IRTlog p(z|x): 0.5174 -> 1.2824Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy