Research map / Probabilistic and scientific modeling / Variational and generative sampling

Environment-Robust Representation Learning with Empirical Bayes

NeurIPS 2026 · Probabilistic and scientific modeling › Variational and generative sampling · approach: Hierarchical VAEs and posterior collapse · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes EBER, an empirical-Bayes amortized variational method for multi-environment prediction where environments shift only the latent-variable distribution, and shows the objective decomposes into per-environment terms plus a cross-environment balancing term.

Problem

Predicting outcomes for observations from unseen environments when environments change the prevalence of latent states while covariate and label mechanisms stay stable conditional on the latent variable.

Approach

Model environments as shifting p(z|e) with stable p(x|z) and p(y|z); derive a variational objective for the marginal Bayes rule p(y|x) whose balancing penalty is induced by the latent model itself, and learn observation-specific environment weights p(e|x) that aggregate environment-specific latent priors at test time.

Key results

• In the parametric simulation, EBER reaches NLL 0.319 and accuracy 0.928, close to the Bayes-x oracle (0.233 NLL, 0.934 accuracy) and better than ERM (0.652 NLL, 0.577 accuracy).
• On PhysioNet sepsis prediction, EBER achieves AUROC 0.740 and AUPRC 0.179, versus 0.724 AUROC for IRM and 0.692 for ERM.
• On quasar-star classification, EBER reaches accuracy 0.921 and NLL 0.221, versus 0.897 accuracy and 0.390 NLL for ERM.

Surprising findings

• Ablations show that using uniform environment weights at test time drops performance to the ERM/IRM range even with oracle posteriors, indicating aggregation weights matter most.
• Naive empirical-frequency environment weights collapse the cross-environment term to zero, so the objective reduces to averaged per-environment terms and fails at test time.

Limitations

• Computing aggregate distributions q(z|x,y) and q(z|x) over all training environments increases computational cost.
• Gains over competitors are not uniform: EBER is not significantly better than VREx on microbiome and than IRM on sepsis.
• Synthetic experiments are designed to match the assumed generative structure, so their results may not transfer to misspecified settings.

Experimental setup

Datasets: Sloan Digital Sky Survey MicrobiomeHD PhysioNet 2019 Challenge colored MNIST (modified)

Benchmarks: parametric simulation colored MNIST simulation SDSS quasar-star classification MicrobiomeHD colorectal cancer classification PhysioNet 2019 sepsis prediction

Metrics: NLL accuracy AUROC AUPRC ECE

Baselines: ERM IRM V-REx Fishr Bayes-x

Builds on: invariant risk minimization V-REx Fishr colored MNIST VampPrior

Closest papers by meaning

• Identifying Causal Direction via Variational Bayesian Compression (ICML 2025 · similarity 0.85)
• Recursive PAC-Bayes: A Frequentist Approach to Sequential Prior Updates with No Information Loss (NeurIPS 2024 · similarity 0.84)
• SoftCVI: Contrastive variational inference with self-generated soft labels (ICLR 2025 · similarity 0.84)
• Causal Representation Learning from Multimodal Biomedical Observations (ICLR 2025 · similarity 0.84)
• Variational Learning of Fractional Posteriors (ICML 2025 · similarity 0.84)
• Censor Dependent Variational Inference (ICML 2025 · similarity 0.84)