Research map / Probabilistic and scientific modeling / Variational and generative sampling

Automated Efficient Estimation using Monte Carlo Efficient Influence Functions

NeurIPS 2024 · Probabilistic and scientific modeling › Variational and generative sampling · approach: Importance sampling and quadrature · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MC-EIF, a fully automated Monte Carlo method that approximates efficient influence functions in parametric models from score-function and Fisher-information quantities available in autodiff/PPL systems, yielding automated one-step, DML and TMLE estimators.

Problem

Efficient influence function (EIF) based estimators require complex, case-by-case analytic derivations, which blocks automation of efficient statistical estimation for high-dimensional parametric models and new functionals.

Approach

Approximate the EIF as the gradient of the functional times the inverse Fisher information times the score, replacing the functional gradient and Fisher information with Monte Carlo estimates computed via autodiff, with conjugate gradient and Pearlmutter's trick for Fisher-vector products.

Key results

• On the minimum variance portfolio task, one-step MC-EIF achieves REV 1.86 ± 0.35 and RMSE 0.08 ± 0.02 versus plug-in REV 2.60 ± 0.35 and RMSE 0.14 ± 0.02.
• MC-EIF with M=10^4 samples approximates the efficient influence function closely in the Gaussian expected-density example, while the empirical Gateaux method is highly sensitive to its two kernel hyperparameters.
• Across ATE estimators (one-step TMLE, DML, one-step), MC-EIF estimates lie close to the diagonal versus analytic-EIF estimates, with the one-step estimator giving over 30% improvement relative to the plug-in.

Measured effects reported in the paper

Surprising findings

• The empirical Gateaux baseline could not achieve numerically stable solutions for any p>2 without prohibitively long run-times.
• DML performed worst in the ATE experiments, possibly because cross-fitting was not used.

Limitations

• Requires the model to be differentiable and simulable (log-densities, scores), and the functional to be differentiable via Monte Carlo gradient estimators.
• Error bounds grow with model dimension p unless M scales as p log p; analysis restricted to parametric models.
• Extensions to latent-variable models and infinite-dimensional targets are only sketched, not evaluated.

Experimental setup

Datasets: simulated Gaussian data simulated confounded treatment-response data (200 confounders, 500 training points) simulated multivariate Gaussian asset returns (D=25, N=1000)

Benchmarks: expected density functional estimation (Gaussian) average treatment effect simulations Markowitz minimum variance portfolio

Metrics: approximation error vs analytic EIF ATE estimate distribution relative expected volatility (REV) RMSE of portfolio weights

Baselines: Empirical Gateaux method analytic EIF plug-in estimator

Builds on: Pyro double machine learning targeted maximum likelihood estimation Pearlmutter trick

Closest papers by meaning

• A Versatile Influence Function for Data Attribution with Non-Decomposable Loss (ICML 2025 · similarity 0.84)
• Decision Making under the Exponential Family: Distributionally Robust Optimisation with Bayesian Ambiguity Sets (ICML 2025 · similarity 0.84)
• Fearless Stochasticity in Expectation Propagation (NeurIPS 2024 · similarity 0.83)
• Latent Variable Estimation in Bayesian Black-Litterman Models (ICML 2025 · similarity 0.82)
• One Sample Fits All: Approximating All Probabilistic Values Simultaneously and Efficiently (NeurIPS 2024 · similarity 0.82)
• On the Tightness and Computational Tractability of Higher-Dimensional Confidence Sequences (NeurIPS 2026 · similarity 0.82)