Research map / Federated learning and privacy / Differential privacy
The Test of Tests: A Framework for Differentially Private Hypothesis Testing
Zeki Kazan, Kaiyan Shi, Adam Groce, Andrew Bray
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Awan-Slavkovic UMP binomial test as aggregation step in subsample-and-aggregate (ToT) instead of PB's randomized-response aggregation | private hypothesis testing framework · Power comparison with PB framework (t-test) | power / sample size for 80% power: n=200 vs n=280 for 80% power at eps=1 | Improved |
| Use Tulap-based binomial aggregation with tuned number of subsamples m and subtest threshold alpha_0 (optimization favoring large m) | Test of Tests · Multivariate normal mean test, d=100, eps=1 | sample size for 80% power: n=65 vs n=6500 for Canonne et al. | Improved |
| Apply ToT framework (subsample-and-aggregate with private binomial test) at eps=1 with exact public test power | generic public tests · Theoretical sample-size multiplier, alpha=0.05 | data multiplier upper bound | No clear effect |
| Tune m and alpha_0 for ToT with a z-test rather than using the upper bound (ablation) | Test of Tests (z-test) · z-test alpha=0.05, effect size 0.65 sd, eps=1 | sample size for 80% power: n=70 vs n=100 bound | Improved |
| Tune m and alpha_0 for ToT on ANOVA instead of Table 1 upper bound (ablation) | Test of Tests (ANOVA, 3 groups) · eps=0.01, 95% power | data multiplier: 41x vs 52x upper bound | Improved |
| Use ToT rather than the Canonne et al. multivariate test (with tuned alpha matched to Type I error) | multivariate normal mean test · d=100, eps=1, small effect mu_1=0.5 | sample size for 80% power: 190 vs 850 | Improved |
| Use ToT rather than Couch et al. nonparametric private ANOVA test | one-way ANOVA, eta=1, eps=1, g=3 · ANOVA power comparison | data needed for 80% power: Couch et al. require 22% less data | Hurt |
| Use ToT rather than Swanberg et al. ANOVA test | one-way ANOVA, eta=1, eps=1, g=3 · ANOVA power comparison | power | Improved |
| Use ToT rather than Couch et al. ANOVA test in smaller effect size setting | one-way ANOVA · small effect size, g=3 | power | Improved |
| Use ToT rather than Couch et al. ANOVA test in large effect size and/or smaller eps settings | one-way ANOVA · large effect size or eps=0.1 | power | Mixed |
| Use ToT rather than Canonne et al. multivariate test at 99% power (high power target) | multivariate normal mean test, small effect · 99% power | sample size needed: ToT needs 87% of Canonne data | Improved |
| Optimize ToT parameters m, alpha_0 with a high-m, high-alpha_0 setting rather than minimal m (PB's low-power recommendation) (ablation) | Peña-Barrientos framework comparison · t-test eps=0.1, mu=3 | sample size for 80% power: n=125 vs n=348 | Improved |
Closest papers by meaning
- Priv’IT: Private and Sample Efficient Identity Testing (ICML 2017 · similarity 0.92)
- Differentially Private Identity and Equivalence Testing of Discrete Distributions (ICML 2018 · similarity 0.91)
- Minimax Optimal Two-Sample Testing under Local Differential Privacy (NeurIPS 2026 · similarity 0.89)
- Sort, Partition, Randomize: Optimal Binary Hypothesis Testing under Local Differential Privacy (NeurIPS 2026 · similarity 0.88)
- Optimal Rates for Differentially Private Hypothesis Testing with E-values (NeurIPS 2026 · similarity 0.88)
- Optimal Private and Communication Constraint Distributed Goodness-of-Fit Testing for Discrete Distributions in the Large Sample Regime (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
