Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Easy Variational Inference for Categorical Models via an Independent Binary Approximation
Michael T Wojnowicz, Shuchin Aeron, Eric L Miller, Michael Hughes
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Bayesian model averaging over CBM and CBC predictions instead of choosing one likelihood | IB-CAVI with CB-Probit or CB-Logit · Simulated categorical prediction | KL divergence to true probabilities | Improved |
| Use CBM rather than CBC as the prediction likelihood (ablation) | IB-CAVI predictions · Simulated data with weakly predictable outcomes | KL divergence to true probabilities | Improved |
| Use CBC rather than CBM as the prediction likelihood (ablation) | IB-CAVI predictions · Simulated data with strongly predictable outcomes | KL divergence to true probabilities | Improved |
| Use IB-Probit-CAVI with CB-Probit and BMA rather than softmax MLE | Categorical regression · Simulated softmax data with few observations relative to parameters | holdout log likelihood | Improved |
| Use IB-Probit-CAVI with CB-Probit and BMA rather than softmax MLE | Categorical regression · Simulated softmax data with abundant observations | holdout log likelihood | No clear effect |
| Use IB-CAVI rather than ADVI for softmax models | Categorical regression · Intrusion detection, 32 users and 1,553 processes | intrusion detection score: 20 minutes per user vs. 200 minutes per user | Improved |
| Use IB-CAVI rather than ADVI, NUTS, or Gibbs sampling | Categorical regression · Detergent, computer process starts, and simulated datasets | training time to prediction quality | Improved |
| Use Bayesian inference with IB-CAVI rather than point-estimated softmax MLE | CB-Probit categorical regression · Simulated data generated by softmax regression | KL divergence between predicted and true probabilities: less than 0.10 | Improved |
| Fit separate independent binary regressions across categories | IB-CAVI · Categorical regression with many categories | scalability: K=1,553 | Improved |
| Use a single IB-CAVI posterior to average over multiple CB prediction likelihoods | CBM and CBC · Categorical prediction | training cost: no additional training cost | Improved |
| Use the IB approximation as a target for CBM rather than CBC (ablation) | IB-Probit approximation · Simulated data, smaller dataset (K=3, M=1, N=1800) | approximation error | Improved |
| Use the IB approximation as a target for CBC rather than CBM (ablation) | IB-Probit approximation · Simulated data, larger dataset (K=20, M=40, N=12000) | approximation error | Improved |
Closest papers by meaning
- PASS-GLM: polynomial approximate sufficient statistics for scalable Bayesian GLM inference (NeurIPS 2017 · similarity 0.86)
- LR-GLM: High-Dimensional Bayesian Inference Using Low-Rank Data Approximations (ICML 2019 · similarity 0.86)
- Differentiable Sampling of Categorical Distributions Using the CatLog-Derivative Trick (NeurIPS 2023 · similarity 0.86)
- Streamlined Variational Inference with Higher Level Random Effects (JMLR 2020 · similarity 0.86)
- Variational Inference with Mixtures of Isotropic Gaussians (NeurIPS 2025 · similarity 0.84)
- Least squares variational inference (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
