Research map / Probabilistic and scientific modeling / Variational and generative sampling
Sourcerer: Sample-based Maximum Entropy Source Distribution Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Sourcerer, a sample-based method that estimates the maximum-entropy source distribution of simulator parameters by minimizing Sliced-Wasserstein distance to observations while annealing an entropy regularizer, and applies it to Hodgkin-Huxley fits on real neuron recordings.
Problem
Source distribution estimation (finding a parameter distribution whose simulations match an observed data distribution) is ill-posed because many sources yield the same pushforward, and existing empirical Bayes methods need tractable or surrogate likelihoods.
Approach
Target the maximum-entropy source among all data-consistent sources, optimize a neural sampler with a Sliced-Wasserstein data-consistency loss plus a Kozachenko-Leonenko entropy estimate, and decay the entropy weight lambda linearly from 1 to a final value during training. The method uses only samples, so it applies to simulators with intractable likelihoods.
Key results
- On the IK benchmark, Sourcerer with entropy regularization reaches entropy 3.75 at C2ST 0.51, versus 0.82 entropy with NEB at C2ST 0.6.
- On the Lotka-Volterra model, entropy regularization raises entropy from -1.34 to 0.29 while keeping C2ST at 0.57.
- On the Hodgkin-Huxley task, the estimated source achieves C2ST of about 61% and an SWD of about 0.08 in standardized summary-statistic space.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add entropy regularization (ablation) | Sourcerer · TM | Entropy: 1.0 (0.198) -> 1.26 (0.022) | Helped |
| add entropy regularization (ablation) | Sourcerer · IK | Entropy: 1.59 (0.246) -> 3.75 (0.066) | Helped |
| add entropy regularization (ablation) | Sourcerer · SLCP | Entropy: 7.23 (0.052) -> 9.81 (0.039) | Helped |
| add entropy regularization (ablation) | Sourcerer · GM | Entropy: -1.25 (0.106) -> -1.12 (0.083) | Helped |
| add entropy regularization (ablation) | Sourcerer · SIR | Entropy: -2.37 (0.169) -> -2.3 (0.079) | No clear effect |
| add entropy regularization (ablation) | Sourcerer · Lotka-Volterra | Entropy: -1.34 (0.087) -> 0.29 (0.017) | Helped |
| add entropy regularization (ablation) | Sourcerer · TM | C2ST accuracy: 0.5 (0.008) -> 0.51 (0.004) | No clear effect |
| add entropy regularization (ablation) | Sourcerer · IK | C2ST accuracy: 0.51 (0.005) -> 0.51 (0.002) | No clear effect |
| add entropy regularization (ablation) | Sourcerer · SLCP | C2ST accuracy: 0.53 (0.006) -> 0.53 (0.005) | No clear effect |
| add entropy regularization (ablation) | Sourcerer · GM | C2ST accuracy: 0.5 (0.006) -> 0.51 (0.005) | No clear effect |
| add entropy regularization (ablation) | Sourcerer · SIR | C2ST accuracy: 0.56 (0.015) -> 0.56 (0.013) | No clear effect |
| add entropy regularization (ablation) | Sourcerer · Lotka-Volterra | C2ST accuracy: 0.52 (0.001) -> 0.57 (0.009) | Mixed |
Surprising findings
- On the deterministic SIR model, entropy regularization does not increase entropy, plausibly because the parameter space has no degeneracy.
- Without entropy regularization, NEB and unregularized SWD sources can ignore large regions of parameter space, introducing severe bias on the Hodgkin-Huxley data.
Limitations
- Results depend on the choice of final regularization strength lambda and on the chosen distance metric.
- Requires a differentiable simulator, often needing a surrogate for discrete or partially discrete simulators.
- Enforces maximum entropy over the full parameter distribution, which may not match other priors such as Jeffreys prior.
Experimental setup
Datasets: Two Moons Inverse Kinematics SLCP Gaussian Mixture SIR Lotka-Volterra Hodgkin-Huxley motor cortex recordings (1033 recordings)
Benchmarks: Two Moons Inverse Kinematics SLCP Gaussian Mixture SIR Lotka-Volterra Hodgkin-Huxley
Metrics: C2ST accuracy entropy of estimated source Sliced-Wasserstein distance
Baselines: Neural Empirical Bayes (NEB) uniform box prior
Builds on: Neural Empirical Bayes Kozachenko-Leonenko entropy estimator RealNVP Maximum Mean Discrepancy
Closest papers by meaning
- A Data-Driven Prism: Multi-View Source Separation with Diffusion Model Priors (NeurIPS 2025 · similarity 0.84)
- Outsourced Diffusion Sampling: Efficient Posterior Inference in Latent Spaces of Generative Models (ICML 2025 · similarity 0.84)
- EVODiff: Entropy-aware Variance Optimized Diffusion Inference (NeurIPS 2025 · similarity 0.84)
- Normative Networks for Source Separation via Local Plasticity and Dendritic Computation (NeurIPS 2026 · similarity 0.84)
- Simultaneous identification of models and parameters of scientific simulators (ICML 2024 · similarity 0.84)
- CCS: Controllable and Constrained Sampling with Diffusion Models via Initial Noise Perturbation (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.