Research map / Probabilistic and scientific modeling / Variational and generative sampling
Direct Fisher Score Estimation for Likelihood Maximization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FSM-MLE, a sequential gradient-based maximum likelihood method for simulator models that estimates the Fisher score via local score matching with a linear, closed-form least-squares surrogate, and links it to Gaussian smoothing.
Problem
Maximum likelihood estimation when the likelihood is intractable but simulations are available (simulation-based inference), where gradient-based MLE needs the Fisher score.
Approach
Adapt score matching to directly model the Fisher score with respect to parameters, using simulations from a Gaussian proposal localized around each parameter iterate; a linear score model gives a closed-form ridge least-squares estimator. The Bayes-optimal score equals the gradient of a Gaussian-smoothed likelihood, so the method is a likelihood-free analogue of Gaussian smoothing.
Key results
- On a multivariate Gaussian with known Fisher score, FSM matches KDE-SP gradient accuracy given sufficient simulation budget, even with untuned hyperparameters.
- On the LSST weak lensing model, sequential gradient-based methods outperform NLE under the same simulation budget, with FSM achieving better performance and smaller variance than KDE-SP.
- On MNIST generator inversion, FSM and direct optimization recover the target observation while KDE-SP struggles with pixel quality.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FSM instead of KDE-SP for gradient estimation | FSM-MLE · 2D Gaussian model | gradient approximation accuracy | Mixed |
| Use FSM instead of KDE-SP as parameter dimension increases at fixed simulation budget | parameter estimation · Multivariate Gaussian model | parameter estimation accuracy | Helped |
| Use sequential FSM or KDE-SP instead of NLE under the same limited simulation budget | LSST weak-lensing cosmology model · LSST-Y10 | parameter estimation and prediction accuracy | Mixed |
| Use FSM or direct optimization instead of KDE-SP for latent inversion | generator inversion · 16×16 MNIST | reconstruction quality | Mixed |
| Vary the proposal variance σ² (ablation) | linear FSM estimator · Gaussian likelihoods with differing curvature | mean-squared error | Mixed |
| Replace the linear score model with a neural network score model (ablation) | FSM · Experiments (appendix comparisons) | empirical performance | Hurt |
| Anneal σ during FSM-MLE instead of using fixed σ (ablation) | FSM-MLE · Experiments | performance | No clear effect |
Surprising findings
- A linear score model is found to work well empirically compared with a neural-network score model, which incurs inner-loop cost and higher variance.
- An annealing schedule for the proposal variance sigma gave no meaningful gains over a fixed sigma in the authors' experiments.
Limitations
- Isotropic Gaussian proposal is suboptimal under strong anisotropic curvature; choosing sigma relies on a grid-search pilot calibration.
- Linear score model may not capture the full data-parameter relationship; neural alternatives are costlier.
- Bias from Gaussian smoothing grows with sigma, creating a bias-variance trade-off; theoretical bias bound relies on assumptions such as L-smoothness and a unique MLE.
Experimental setup
Datasets: simulated multivariate Gaussian data LSST-Y10 log-normal weak lensing simulations 16x16 MNIST
Benchmarks: multivariate Gaussian MLE gradient accuracy LSST weak lensing parameter estimation generator inversion on MNIST
Metrics: gradient estimation error MSE of parameter estimate reconstruction loss standard deviation across runs
Baselines: KDE-SP NLE (SNLE) direct optimization
Builds on: score matching Gaussian smoothing gradient estimator KDE-SP approximate MLE SNLE Polyak-Ruppert averaging
Closest papers by meaning
- Bayesian Optimization with Fisher Information Geometry: Gradient Bounds and Trust-Region Methods (NeurIPS 2026 · similarity 0.85)
- Fishers for Free? Approximating the Fisher Information Matrix by Recycling the Squared Gradient Accumulator (ICML 2025 · similarity 0.85)
- Sequential Neural Score Estimation: Likelihood-Free Inference with Conditional Score Based Diffusion Models (ICML 2024 · similarity 0.84)
- Bayesian Optimisation with Unknown Hyperparameters: Regret Bounds Logarithmically Closer to Optimal (NeurIPS 2024 · similarity 0.84)
- Direct Amortized Likelihood Ratio Estimation (AAAI 2024 · similarity 0.84)
- Scale-invariant Optimal Sampling for Rare-events Data and Sparse Models (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.