Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
A Metalearned Neural Circuit for Nonparametric Bayesian Inference
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Metalearns a GRU sequence model on sequences sampled from a CRP/DPMM prior so it performs sequential nonparametric Bayesian inference over an open set of classes, comparing against particle filter baselines.
Problem
Exact and approximate sequential inference in Dirichlet process mixture models (DPMMs) is computationally expensive and restricted to exponential-family likelihoods, limiting practical use for open-set classification with complex inputs.
Approach
Train an RNN (a "neural circuit") by metalearning on sequences sampled from a CRP/DPMM prior, minimizing next-label negative log-likelihood, so the network amortizes the DPMM posterior predictive without explicit particles or conjugacy assumptions; the output head is masked so only valid new-class labels are allowed.
Key results
- On synthetic DPMM data, the neural circuit achieves ARI 0.9225 in the fully unobserved setting vs 0.7691 for the particle filter, while the particle filter has better NLL (0.0484 vs 0.0746).
- On ImageNet-CRP meta-test classes, the neural circuit reaches NLL 0.6639 and ARI 0.2802 vs particle filter NLL 0.9333 and ARI 0.0481.
- Fully unobserved inference is roughly 10x faster than the particle filter on synthetic data and over 100x faster on ImageNet-CRP (0.50 ms vs 73.9 ms per sequence).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the Neural Circuit instead of the Particle Filter | DPMM sequential inference · Synthetic DPMM | NLL: 0.0484 -> 0.0746 | Hurt |
| Use the Neural Circuit instead of the Particle Filter | DPMM sequential inference · Synthetic DPMM | ARI: 0.7691 -> 0.9225 | Helped |
| Use the Neural Circuit instead of the Particle Filter | DPMM sequential inference · Synthetic DPMM | inference time (ms/sequence), sequential observation: 1.6168 -> 0.3153 | Helped |
| Use the Neural Circuit instead of the Particle Filter | DPMM sequential inference · ImageNet-CRP meta-train classes | NLL: 0.8478 -> 0.2331 | Helped |
| Use the Neural Circuit instead of the Particle Filter | DPMM sequential inference · ImageNet-CRP meta-train classes | ARI: 0.0699 -> 0.7720 | Helped |
| Use the Neural Circuit instead of the Particle Filter | DPMM sequential inference · ImageNet-CRP meta-test classes | NLL: 0.9333 -> 0.6639 | Helped |
| Use the Neural Circuit instead of the Particle Filter | DPMM sequential inference · ImageNet-CRP meta-test classes | ARI: 0.0481 -> 0.2802 | Helped |
| Use the Neural Circuit instead of the Particle Filter | DPMM sequential inference · ImageNet-CRP fully unobserved | inference time (ms/sequence): 73.8964 -> 0.5013 | Helped |
Surprising findings
- The particle filter achieves better sequential-observation NLL on synthetic data, yet the neural circuit gives much better clusterings in the fully unobserved setting.
Limitations
- Particle filter baseline uses true hyperparameters on synthetic data and a hurdle log-normal model adapted to ReLU features, so comparisons depend on those choices.
- Performance drops notably on meta-test classes not seen during metalearning (ARI 0.28 vs 0.77 on meta-train).
- Only one random seed per method was used.
Experimental setup
Datasets: ILSVRC 2012 (ImageNet) ResNet-18 features synthetic normal-inverse-gamma DPMM sequences
Benchmarks: synthetic DPMM sequences ImageNet-CRP
Metrics: NLL perplexity ARI AMI wall-clock inference time per sequence
Baselines: CRP Particle Filter
Builds on: Dirichlet process mixture model Chinese restaurant process particle filter for DPMM (Ahn et al.) FLOWR
Closest papers by meaning
- Bayesian Low-Rank Learning (Bella): A Practical Approach to Bayesian Neural Networks (AAAI 2025 · similarity 0.84)
- Do Bayesian Neural Networks Actually Behave Like Bayesian Models? (ICML 2025 · similarity 0.84)
- A Study of Bayesian Neural Network Surrogates for Bayesian Optimization (ICLR 2024 · similarity 0.84)
- DNA-SE: Towards Deep Neural-Nets Assisted Semiparametric Estimation (ICML 2024 · similarity 0.83)
- Leveraging Self-Consistency for Data-Efficient Amortized Bayesian Inference (ICML 2024 · similarity 0.83)
- Scalable Bayesian Learning with posteriors (ICLR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.