Research map / Probabilistic and scientific modeling / Differential equations and PDEs
DNA-SE: Towards Deep Neural-Nets Assisted Semiparametric Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DNA-SE, which casts semiparametric estimation as a bi-level optimization and uses deep feedforward networks with alternating gradient descent to numerically solve the Fredholm integral equations that arise in efficient estimators.
Problem
Semiparametric efficient estimators often require numerically solving Fredholm integral equations of the second kind that depend on the parameter of interest; traditional polynomial/spline solvers scale poorly to multi-dimensional settings, while closed-form alternatives are statistically suboptimal
Approach
Treat a deep feedforward network as the solution of the integral equation and jointly estimate it with the parameter of interest via a nested (bi-level) loss optimized by alternating gradient descent, using Monte Carlo integration.
Key results
- In the MNAR simulation (N=500, 100 reps), DNA-SE gives beta1 bias 0.012 (sd 0.142) vs. 0.070-0.186 sd for polynomial baselines.
- In transfer learning simulation (N=10000), DNA-SE gives bias -0.0008 (sd 0.0084) vs. quintic polynomial sd 0.0186, reducing variance by at least 80%.
- On the real CBCL data, DNA-SE gives beta_z = 1.4620 with se 0.0761, smaller than the 0.1194 reported by the prior analysis.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DNA-SE instead of polynomial approximations | MNAR regression parameter estimation · MNAR simulation | bias: DNA-SE 0.012(0.142); quintic -0.113(0.070); quartic -0.081(0.186); cubic -0.143(0.133); quadratic -0.033(0.178) | Helped |
| Use DNA-SE instead of the polynomial method [cite] | MNAR regression parameter estimation · MNAR simulation | variance: DNA-SE 0.012(0.142); [cite] -0.016(0.209) | Helped |
| Use DNA-SE instead of polynomial approximations | Causal sensitivity analysis · Causal sensitivity simulation | bias: DNA-SE 0.036(0.155); cubic 0.142(0.189); quadratic 0.145(0.188) | Helped |
| Use DNA-SE instead of the grid-based method | Causal sensitivity analysis · Causal sensitivity simulation | variance: DNA-SE 0.036(0.155); grid-based method 0.109(0.426) | Helped |
| Use DNA-SE instead of polynomial approximations | Transfer learning average-loss estimation · Transfer learning simulation | variance: DNA-SE -0.0008(0.0084); quintic 0.0009(0.0186); 10th-degree 0.0033(0.0234); 15th-degree 0.0038(0.0385) | Helped |
| Use DNA-SE instead of the observed-only estimator | Connecticut children's mental health study · Real-data regression coefficients other than βz | estimate: β0: -1.3289(0.1402) vs -1.9307(0.1132); β1: -0.0623(0.1196) vs 0.3652(0.1480); β2: -0.6159(0.1366) vs -0.0516(0.1690) | Helped |
| Use DNA-SE instead of method [cite] | Connecticut children's mental health study · Real-data regression coefficients | standard error: β0: 0.1402 vs 0.1823; β1: 0.1196 vs 0.1470; βz: 0.0761 vs 0.1194 | Helped |
| Set alternating frequency γ to 1 (ablation) | DNA-SE for MNAR regression estimation · MNAR simulation | convergence stability | Hurt |
| Increase alternating frequency above 5 (ablation) | DNA-SE for MNAR regression estimation · MNAR simulation | convergence | Mixed |
| Vary alternating frequency γ among 1, 5, 10, and 20 (ablation) | DNA-SE for causal sensitivity analysis · Causal sensitivity simulation | convergence rate | Mixed |
| Vary Monte Carlo sample sizes J1 and J2 while keeping them sufficiently large (ablation) | DNA-SE · MNAR simulation | results | No clear effect |
Surprising findings
- The alternating frequency gamma=1 fails to converge in the MNAR example, oscillating at high frequency.
Limitations
- No convergence or asymptotic theory is provided; the authors deliberately leave the statistical analysis of the training algorithm to future work.
- Results depend on hyperparameter tuning (width, depth, alternating frequency) and the global minimum is not guaranteed for the bi-level objective.
- Experiments are small proof-of-concept simulations with a single real dataset and bootstrap-based standard errors.
Experimental setup
Datasets: simulated data from three semiparametric DGPs Connecticut Children's Mental Health Study
Benchmarks: simulated MNAR regression simulated sensitivity analysis causal inference simulated transfer learning average loss Connecticut Children's Mental Health Study
Metrics: bias (mean of estimate minus truth) standard deviation standard error
Baselines: polynomial approximation (quadratic to quintic) grid-based method observed-only estimator Miao et al. shadow-variable estimator
Builds on: shadow variable approach to MNAR semiparametric efficient estimation Tikhonov-regularized sensitivity analysis covariate shift posterior drift
Closest papers by meaning
- A Metalearned Neural Circuit for Nonparametric Bayesian Inference (NeurIPS 2024 · similarity 0.83)
- Nonparametric Estimation of a Factorizable Density using Diffusion Models (NeurIPS 2026 · similarity 0.83)
- Scalable Discrete Diffusion Samplers: Combinatorial Optimization and Statistical Physics (ICLR 2025 · similarity 0.83)
- Sequential Neural Score Estimation: Likelihood-Free Inference with Conditional Score Based Diffusion Models (ICML 2024 · similarity 0.83)
- Can Neural Networks Achieve Optimal Computational-statistical Tradeoff? An Analysis on Single-Index Model (ICLR 2025 · similarity 0.83)
- Towards Self-Supervised Covariance Estimation in Deep Heteroscedastic Regression (ICLR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.