Research map / Probabilistic and scientific modeling / Variational and generative sampling
Variational Memory Addressing in Generative Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use stochastic discrete memory addressing with variational inference instead of soft attention | Omniglot generative few-shot learning · Omniglot few-shot | NLL: 90.8 -> 68.8 | Helped |
| Use stochastic discrete memory addressing instead of soft attention | Omniglot few-shot learning · Omniglot with increasing confounding character classes | NLL: Generative Matching Nets: 83.3 to 45.8; Variational Memory Addressing: 86.5 to 73.9 | Mixed |
| Increase examples per class in memory | Variational Memory Addressing · Omniglot few-shot | test NLL: 75.1 to 69.1 nats | Helped |
| Increase memory examples per class, shifting modeling responsibility toward discrete addresses | Variational Memory Addressing · Omniglot few-shot | KL(q(a|x) || p(a)): 2.0 to 4.5 nats; continuous KL 28.2 to 21.8 nats | Mixed |
| Use a flat address prior instead of a learned prior (ablation) | Variational Memory Addressing · Generative experiments | performance | Hurt |
| Replace the normalized inner-product similarity with plain inner product or cosine similarity (ablation) | Variational Memory Addressing · Memory address inference | performance | Hurt |
| Tie the embedding parameters for the prior and posterior address distributions (ablation) | Variational Memory Addressing · Omniglot | performance | No clear effect |
| Increase memory capacity to 2048 without regularization (ablation) | Variational Memory Addressing · Omniglot learned memory | testset NLL: training-set NLL < 80 nats; testset NLL > 150 nats | Hurt |
| Use discrete memory addressing rather than a continuous latent model alone | Variational Memory Addressing · MNIST | final NLL | No clear effect |
| Use a larger memory at test time than during training | Variational Memory Addressing · Omniglot few-shot | NLL relative to optimal model | No clear effect |
| Constrain memory size to 256 and add L2 weight decay (ablation) | Variational Memory Addressing · Omniglot learned memory | test NLL: 103.6 nats | Helped |
Closest papers by meaning
- Hierarchical Variational Memory for Few-shot Learning Across Domains (ICLR 2022 · similarity 0.89)
- Deep Attentive Variational Inference (ICLR 2022 · similarity 0.88)
- On the relationship between variational inference and auto-associative memory (NeurIPS 2022 · similarity 0.88)
- From Variational to Deterministic Autoencoders (ICLR 2020 · similarity 0.88)
- Semi-Amortized Variational Autoencoders (ICML 2018 · similarity 0.88)
- InfoVAE: Balancing Learning and Inference in Variational Autoencoders (AAAI 2019 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.