Research map / Learning theory and algorithms / Sample complexity
Estimating Entropy of Distributions in Constant Space
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use multiple probability intervals, increasing to T = \log^* k (ablation) | Constant-space entropy estimation · k-ary distribution entropy estimation | sample complexity: O\left(\frac{k\log^2 \left(k/\varepsilon\right)}{\varepsilon^3}\right) -> O({k (\log (1/\varepsilon))^2}/{\varepsilon^3} | Helped |
| Use two probability intervals instead of one (ablation) | Constant-space entropy estimation · k-ary distribution entropy estimation | sample complexity: O\left(\frac{k\log^2 \left(k/\varepsilon\right)}{\varepsilon^3}\right) -> O\left( \frac{k (\log \left(\log (k)/\varepsil | Helped |
| Use constant memory rather than the sample-optimal implementation | General Intervals Algorithm · k-ary distribution entropy estimation | space complexity (words): O\left(\frac{k}{\varepsilon\log k}+\frac{\log^2 k}{\varepsilon^2}\right) -> 20 | Helped |
| Use the constant-space interval algorithm rather than a sample-optimal algorithm | Entropy estimation · k-ary distribution entropy estimation | sample complexity: \Theta\left(\frac{k}{\varepsilon\log k}+\frac{\log^2 k}{\varepsilon^2}\right) -> O({k (\log (1/\varepsilon))^2}/{\vareps | Hurt |
| Clip estimates in the smallest-probability interval (ablation) | Two Intervals Algorithm · k-ary distribution entropy estimation | sample complexity: R_2 = O\left( { (\log \left( (\log k)/\varepsilon\right))^2}/{\varepsilon^2}\right) | Helped |
Closest papers by meaning
- Estimation of Entropy in Constant Space with Improved Sample Complexity (NeurIPS 2022 · similarity 0.92)
- Entropy testing and its application to testing Bayesian networks (NeurIPS 2024 · similarity 0.84)
- Sharp Optimality of Simple, Plug-in Estimation of the Fisher Information of a Smoothed Density (ICML 2025 · similarity 0.83)
- Optimal Algorithms for Augmented Testing of Discrete Distributions (NeurIPS 2024 · similarity 0.83)
- Differentially Private Space-Efficient Algorithms for Counting Distinct Elements in the Turnstile Model (ICML 2025 · similarity 0.83)
- Near-Optimal Streaming Heavy-Tailed Statistical Estimation with Clipped SGD (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.