Research map / Learning theory and algorithms / Sample complexity
Dimension-free empirical entropy estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Assume a bounded αth information moment with α>1 and use the plug-in estimator | Entropy estimation on infinite alphabets · Infinite alphabet, bounded αth information moment | L1 minimax risk: \frac{1+\log n}{\sqrt{n}} + \frac{2^{\alpha-1}h}{\log^{\alpha-1}n} | Helped |
| Assume only finite entropy (α=1), rather than an information moment of order greater than one | Entropy estimation on infinite alphabets · Infinite alphabet, finite entropy only | L1 minimax risk: Ch | Hurt |
| Use the empirical plug-in estimate under an αth information-moment assumption with α>1 | Entropy estimation on infinite alphabets · Infinite alphabet, bounded αth information moment | absolute entropy error | Helped |
| Replace the finite-alphabet continuity condition with a bounded information-moment condition | Entropy continuity for discrete distributions · Infinite alphabet | entropy continuity under ℓ1 | Helped |
| Use only a first information moment (α=1) instead of a moment of order greater than one | Entropy continuity for discrete distributions · Infinite alphabet, bounded first information moment | entropy continuity under ℓ1 | Hurt |
| Use empirical rather than worst-case distribution-independent accuracy bounds | Plug-in entropy estimator · Natural finite- and infinite-support distributions | entropy error bound | Mixed |
Closest papers by meaning
- Estimation of Entropy in Constant Space with Improved Sample Complexity (NeurIPS 2022 · similarity 0.88)
- Instance-Optimality for Private KL Distribution Estimation (NeurIPS 2025 · similarity 0.85)
- Entropy testing and its application to testing Bayesian networks (NeurIPS 2024 · similarity 0.84)
- The Empirical Mean is Minimax Optimal for Local Glivenko-Cantelli (ICML 2025 · similarity 0.84)
- Sharp Optimality of Simple, Plug-in Estimation of the Fisher Information of a Smoothed Density (ICML 2025 · similarity 0.84)
- Information-Theoretic Discrete Diffusion (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.