Research map / Learning theory and algorithms / Prediction and decision losses
Sequential Kernel Goodness-of-fit Testing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes sequential kernel goodness-of-fit tests (SKGT) built on testing by betting, using KSD, KDSD and bounded-domain KSD payoffs with a wealth process for anytime-valid type-I error control.
Problem
Goodness-of-fit testing on streaming data where the sample size is not fixed in advance; classical batch tests need multiple-testing corrections under continuous monitoring, which reduces power.
Approach
Reframes sequential goodness-of-fit testing as a betting game: a bettor selects kernel Stein discrepancy-based payoff functions (with plug-in witness estimates) and bets fractions chosen by online Newton step, rejecting when wealth exceeds 1/alpha; extends to discrete (KDSD) and bounded domains (bd-KSD).
Key results
- Across 11 mixing values, the KSD-based SKGT required on average fewer samples to reject than batch KSD tests with Bonferroni correction monitored every 10 or 100 points (Figure 1).
- Under the null, the Student's t experiment controls type-I error, and expected stopping time grows as degrees of freedom decrease from inf to 1 (Figure 2).
- The KDSD-based test does not reject within 1000 samples under the Ising null and the bd-KSD test does not reject within 10^4 samples under the truncated Gaussian null.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use KSD-based SKGT instead of batch KSD with Bonferroni correction every 10 or 100 points | KSD-based SKGT · Continuous monitoring simulation | average stopping time | Helped |
| Increase β toward 1 in the mixture distribution | KSD-based SKGT and batch KSD with Bonferroni correction · Continuous monitoring simulation | average stopping time | Mixed |
| Use samples from Student’s t distributions with degrees of freedom farther from 1 | KSD-based SKGT · Student’s t versus Normal | average stopping time | Helped |
| Test the null setting with samples from the null distribution | KSD-based SKGT · Student’s t versus Normal | type-I error | No clear effect |
| Move the Ising model temperature toward the null temperature T0 = 5 | KDSD-based SKGT · Ising model | average stopping time | Hurt |
| Test under the Ising-model null temperature | KDSD-based SKGT · Ising model | rejection within 1000 samples | No clear effect |
| Decrease the mean shift v from 0.9 toward 0 | bd-KSD-based SKGT · Truncated Gaussians in Unit Ball | average stopping time | Hurt |
| Test under the null mean shift v = 0 | bd-KSD-based SKGT · Truncated Gaussians in Unit Ball | rejection within 104 samples | No clear effect |
Limitations
- Sequential tests are described as complements to batch tests rather than replacements; the KSD test can be expensive for large samples
- Theoretical guarantees require assumptions on bounded kernels and score norms (Assumptions 2.3-2.4) and bounded h for bd-KSD
- Experiments are limited to simulated distributions with a single baseline
Experimental setup
Datasets: simulated Student's t samples (df 1, 5, 10, inf) Ising model samples (10x10 lattice, various temperatures) truncated Gaussian samples in the unit ball
Benchmarks: simulated Student's t vs normal Ising model temperature test truncated Gaussian mean-shift test mixture of Gaussian and Cauchy-type target continuous monitoring
Metrics: type-I error control average stopping time power
Baselines: Batch KSD test with Bonferroni correction
Builds on: testing by betting kernel Stein discrepancy Shekhar & Ramdas sequential kernel two-sample test Cutkosky & Orabona coin betting KDSD bounded-domain KSD
Closest papers by meaning
- Peeking with PEAK: Sequential, Nonparametric Composite Hypothesis Tests for Means of Multiple Data Streams (ICML 2024 · similarity 0.86)
- YEAST: Yet Another Sequential Test (NeurIPS 2025 · similarity 0.86)
- Optimal Algorithms for Augmented Testing of Discrete Distributions (NeurIPS 2024 · similarity 0.85)
- Time-Sensitive Anytime-Valid Testing (NeurIPS 2026 · similarity 0.85)
- Kernel Selection is Model Selection: A Unified Complexity-Penalised Approach for MMD Two-Sample Tests (NeurIPS 2026 · similarity 0.84)
- DUAL: Learning Diverse Kernels for Aggregated Two-sample and Independence Testing (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.