Research map / Probabilistic and scientific modeling / Variational and generative sampling
Neural Autoregressive Distribution Estimation
Benigno Uria, Marc-Alexandre Côté, Karol Gregor, Iain Murray, Hugo Larochelle
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| provide mask information about which inputs are observed (ablation) | DeepNADE 2hl · binarized MNIST | test negative log-likelihood: 95.33 -> 89.17 | Improved |
| provide mask information about which inputs are observed (ablation) | ConvNADE+DeepNADE · binarized MNIST | test negative log-likelihood: 85.25 -> 80.82 | Improved |
| add a second hidden layer | DeepNADE · binarized MNIST | test negative log-likelihood: 92.17 -> 89.17 | Improved |
| add more hidden layers (3 or 4) | DeepNADE 2hl · binarized MNIST | test negative log-likelihood: 89.17 -> 89.38, 89.60 | No clear effect |
| ensemble over 128 orderings instead of 2 | EoNADE 2hl · binarized MNIST | test negative log-likelihood: 87.96 -> 85.10 | Improved |
| add convolutional layers | NADE variants · binarized MNIST | test negative log-likelihood: 89.17 -> 81.30 | Improved |
| add a DeepNADE of fully-connected layers on top of ConvNADE | ConvNADE · binarized MNIST | test negative log-likelihood: 81.30 -> 80.82 | Improved |
| use order-agnostic training with ensemble (NADE-E) instead of fixed order | NADE · Binary vector datasets (8) | average log-likelihood: -13.19 -> -13.19 Adult; -12.58 vs -11.99 Connect4; -273.08 -> -272.39 NIPS-0-12 | Mixed |
| replace Gaussian with fixed variance by variable-variance Gaussian conditionals (ablation) | RNADE · BSDS300 8x8 patches | test log-likelihood (nats): 100.3 -> 143.9 | Improved |
| use mixture of Gaussians conditionals instead of single Gaussian (ablation) | RNADE · BSDS300 8x8 patches | test log-likelihood (nats): 143.9 -> 153.5 | Improved |
| increase mixture components K from 2 to 10 (ablation) | RNADE-MoG · TIMIT core-test | test log-likelihood (nats): 121.1 -> 127.8 | Improved |
| increase hidden units from 512 to 1024 | RNADE-MoG K=10 · BSDS300 8x8 patches | test log-likelihood (nats): 153.5 -> 153.7 | Improved |
Closest papers by meaning
- Document Neural Autoregressive Distribution Estimation (JMLR 2017 · similarity 0.87)
- Masked Autoregressive Flow for Density Estimation (NeurIPS 2017 · similarity 0.86)
- Few-shot Autoregressive Density Estimation: Towards Learning to Learn Distributions (ICLR 2018 · similarity 0.86)
- Learning Gaussian Conditional Distributions using Neural Ratio Estimation is Hard (NeurIPS 2026 · similarity 0.85)
- Nonparametric Estimation of a Factorizable Density using Diffusion Models (NeurIPS 2026 · similarity 0.83)
- Fundamental Limits of Visual Autoregressive Transformers: Universal Approximation Abilities (ICML 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
