Research map / Probabilistic and scientific modeling / Variational and generative sampling
A Differentiable Partially Observable Generalized Linear Model with Forward-Backward Message Passing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Makes the partially observable GLM differentiable via Gumbel-Softmax and continuous relaxations so variational inference can use pathwise gradients, and introduces a forward-backward message-passing variational sampling scheme for neural connectivity inference.
Problem
Learning partially observable generalized linear models (POGLM) of neural spike trains with hidden neurons via variational inference is hard because the discrete Poisson hidden spike counts block pathwise gradients, and existing variational sampling schemes are inexpressive and slow.
Approach
Relax discrete hidden spike counts into differentiable variables (Gumbel-Softmax over truncated Poisson, or continuous Exp/Rayleigh/half-normal distributions) so the pathwise gradient estimator applies, and use a forward-backward variational sampler where hidden spikes are conditioned on both past and future visible spikes.
Key results
- On the synthetic data, Gumbel-Softmax with pathwise gradients raises test log-likelihood significantly above the original Poisson POGLM, and continuous Exp distribution with forward-backward sampling gives the best LL and parameter errors.
- On the RGC dataset, Exp x FB is significantly better than all other method combinations in test log-likelihood.
- On PVC-5, the optimal number of hidden neurons is no more than 3 for all method combinations, and differentiable methods show much smaller test LL variance.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace Poisson hidden spikes with categorical, then Gumbel-Softmax (ablation) | POGLM inference · Synthetic dataset | test log-likelihood | Hurt |
| use the pathwise gradient estimator instead of the score-function estimator for Gumbel-Softmax (ablation) | Gumbel-Softmax POGLM · Synthetic dataset | test log-likelihood | Helped |
| replace Gumbel-Softmax with exponential, Rayleigh, or half-normal hidden spike distributions (ablation) | Pathwise-gradient POGLM · Synthetic dataset | test log-likelihood | Helped |
| use forward-backward message passing instead of forward or forward-self sampling (ablation) | POGLM with a good inference method · Synthetic dataset | test log-likelihood | Helped |
| use forward sampling instead of forward-backward or forward-self sampling (ablation) | POGLM without a good inference method · Synthetic dataset | test log-likelihood | Helped |
| use exponential, Rayleigh, or half-normal distributions with pathwise gradients (ablation) | POGLM parameter recovery · Synthetic dataset | parameter error | Helped |
| remove hidden neurons by using a fully observed GLM instead (ablation) | POGLM · RGC | test log-likelihood | Helped |
| use Rayleigh hidden spikes with forward-backward sampling (ablation) | POGLM · RGC | test log-likelihood | Hurt |
| use a differentiable inference method instead of a non-differentiable method (ablation) | POGLM · RGC | test log-likelihood | Helped |
| use exponential hidden spikes with forward-backward sampling (ablation) | POGLM · RGC | test log-likelihood | Helped |
| increase the number of hidden neurons (ablation) | GS-p and exponential POGLM · RGC | test log-likelihood | Helped |
| increase the number of hidden neurons beyond 3 (ablation) | POGLM · PVC-5 | test log-likelihood | Hurt |
Surprising findings
- The posterior seems closer to a continuous distribution (exponential-like) than to the discrete Poisson, so continuous relaxations outperform the faithful Gumbel-Softmax approximation.
- Non-differentiable inference methods with more hidden neurons can perform worse than a fully observed GLM on PVC-5.
Limitations
- Relaxing to continuous distributions loses the spike-count interpretation of Z.
- Hidden spike count truncated at M=5 and temperature tau fixed at 0.5 via limited tuning.
- Experiments are small-scale (5 neurons synthetic, few neurons in real data).
Experimental setup
Datasets: RGC dataset (mouse retinal ganglion cells) PVC-5 (macaque V1 recordings, CRCNS)
Benchmarks: synthetic POGLM dataset RGC dataset PVC-5 dataset
Metrics: test log-likelihood weight estimation error bias estimation error running time
Baselines: Poisson score-function VI Categorical score-function VI Gumbel-Softmax score-function VI fully observed GLM
Builds on: POGLM Gumbel-Softmax variational inference
Closest papers by meaning
- Scalable inference of functional neural connectivity at submillisecond timescales (NeurIPS 2025 · similarity 0.87)
- Poisson Variational Autoencoder (NeurIPS 2024 · similarity 0.85)
- Multi-modal Gaussian Process Variational Autoencoders for Neural and Behavioral Data (ICLR 2024 · similarity 0.84)
- Conditionally-Conjugate Gaussian Process Factor Analysis for Spike Count Data via Data Augmentation (ICML 2024 · similarity 0.84)
- Spike Distance Function as a Learning Objective for Spike Prediction (ICML 2024 · similarity 0.84)
- Identifying interactions across brain areas while accounting for individual-neuron dynamics with a Transformer-based variational autoencoder (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.