Research map / Learning theory and algorithms / Sample complexity
Sharp Optimality of Simple, Plug-in Estimation of the Fisher Information of a Smoothed Density
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Derives the sharp minimax rate for estimating the Fisher information of a Gaussian-smoothed Hölder density and shows simple plug-in estimators attain it for t>0, with consequences for mutual information and entropy estimation.
Problem
Estimating the Fisher information of f*φ_t (Gaussian-smoothed density) from i.i.d. samples of a compactly supported α-Hölder density, including the sharp dependence on the smoothing level t.
Approach
Shows that, unlike the unsmoothed case where debiasing is needed, a truncated plug-in estimator built from kernel/empirical estimates of the density and its derivative attains the minimax rate for t>0; uses a lower-bound construction with an extra tunable parameter ρ (from Dou et al. 2024) and I-MMSE/de Bruijn identities to get parametric-rate mutual information and entropy estimators.
Key results
- Minimax rate for Fisher information of f*φ_t is 1/√(nt^2) ∧ 1/√(nt^{3/4}) ∧ n^{-α/(2α+1)}/√t in the regimes stated, with plug-in upper bounds matching lower bounds for t ≤ c or t ≥ C.
- Plug-in estimators of mutual information and entropy under Gaussian smoothing achieve the parametric rate 1/√n for any fixed t>0 via I-MMSE and de Bruijn identities.
- Minimax rate blows up as t→0 (e.g., ~1/√t for the uniform example), while for t≥1 it decreases in t.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the truncated plug-in estimator in the high-noise regime | Fisher information of f ∗ φt · n^{-2/(2α+1)} < t < 1 | minimax absolute error rate: 1/(√nt^{3/4}) | Helped |
| Use the kernel-based truncated plug-in estimator in the low-noise regime | Fisher information of f ∗ φt · t ≤ n^{-2/(2α+1)} | minimax absolute error rate: n^{-α/(2α+1)}/√t | Helped |
| Use the decomposed plug-in estimator in the very-high-noise regime | Fisher information of f ∗ φt · t ≥ C | minimax absolute error rate: 1/(√nt2) | Helped |
| Estimate mutual information by integrating the Fisher-information estimator via I-MMSE | I(Xt; µ) · Fixed t > 0 | absolute error rate: C/√n | Helped |
| Estimate smoothed entropy from the mutual-information estimator | h(f ∗ φt) · Fixed t > 0 | absolute error rate: C/√n | Helped |
| Estimate unsmoothed entropy using de Bruijn’s identity and the Fisher-information estimator | h(f) · Fixed t > 0 | absolute error rate: C/√n | Helped |
| Decrease the smoothing level t toward zero | Fisher-information estimation · Low-noise regime | minimax absolute error rate: n^{-α/(2α+1)}/√t | Hurt |
| Increase t within t > 0 | Fisher-information estimation · Smoothed density | minimax absolute error rate | Helped |
Surprising findings
- Plug-in estimators are rate-optimal for t>0, even though debiasing is required to attain the sharp rate in the unsmoothed t=0 case.
- Minimax rate for small t>0 is slower than in the unsmoothed setting, so low Gaussian smoothing makes Fisher information estimation harder.
Limitations
- Theory has a gap for c < t < C between upper and lower bounds (conjectured artifact of proof).
- Assumes density bounded below by a constant; the case without this lower bound is left open.
- Purely theoretical; no experiments reported.
Experimental setup
Metrics: minimax rate (expected absolute error)
Baselines: Laurent (1997) Bhattacharya (1967) Cao et al. (2020) Goldfeld et al. (2020b) plug-in entropy estimator
Builds on: Laurent (1997) debiased plug-in Fisher information estimator Goldfeld et al. (2020b) Gaussian smoothing entropy estimation Dou et al. (2024) sharp score matching lower bounds Cao et al. (2020) Fisher information estimator
Closest papers by meaning
- The Nearest Neighbor Information Estimator is Adaptively Near Minimax Rate-Optimal (NeurIPS 2018 · similarity 0.87)
- Optimal Rates for Nonparametric Density Estimation under Communication Constraints (NeurIPS 2021 · similarity 0.86)
- Distributed Nonparametric Estimation: from Sparse to Dense Samples per Terminal (ICML 2025 · similarity 0.84)
- Instance-Optimality for Private KL Distribution Estimation (NeurIPS 2025 · similarity 0.84)
- Near-Optimal Streaming Heavy-Tailed Statistical Estimation with Clipped SGD (NeurIPS 2024 · similarity 0.83)
- Sharp Rates in Dependent Learning Theory: Avoiding Sample Size Deflation for the Square Loss (ICML 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.