Research map / Learning theory and algorithms / Prediction and decision losses
Kernel Stein Tests for Multiple Model Comparison
Jen Ning Lim, Makoto Yamada, Bernhard Schölkopf, Wittawat Jitkrittum
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use RelPSI-MMD instead of selecting the lowest FID score | CelebA image comparison · CelebA | model selection accuracy: 73% | Improved |
| Use relative testing rather than choosing the model with the minimum FID score | CelebA image comparison · CelebA | identification of worse candidates | Improved |
| Use RelPSI-KSD rather than RelMulti-KSD | Chicago Crime density comparison · Chicago Crime | model-wise rejection rate: MoG (1): 0.22 -> 0.42; MoG (2): 0.07 -> 0.28; MADE: 0.04 -> 0.26 | Improved |
| Use FID minimum-score selection rather than relative MMD testing | CelebA image comparison · CelebA | incorrect model selection rate: 73% | Hurt |
| Condition the test on the model-selection event | RelPSI versus unconditioned RelMMD/RelKSD · Toy problems | test power | Hurt |
| Use KSD instead of MMD | Kernel relative tests · Toy problems | test power | Improved |
| Use linear-time estimators instead of complete estimators | Kernel relative tests · Toy problems | test power | Hurt |
| Use the median-heuristic bandwidth for MMD in the blobs setting | MMD-based tests · Blobs | rejection rate | Hurt |
| Use data splitting for reference selection and testing | RelMulti · Multiple model comparison | true positive rate | Hurt |
| Use RelPSI rather than RelMulti under the theorem's stated conditions | Two-model comparison · Two candidate models | true positive rate | Improved |
Closest papers by meaning
- DUAL: Learning Diverse Kernels for Aggregated Two-sample and Independence Testing (NeurIPS 2025 · similarity 0.86)
- Kernel Selection is Model Selection: A Unified Complexity-Penalised Approach for MMD Two-Sample Tests (NeurIPS 2026 · similarity 0.85)
- A General Framework for the Analysis of Kernel-based Tests (JMLR 2024 · similarity 0.85)
- Random Feature Stein Discrepancies (NeurIPS 2018 · similarity 0.85)
- Kernel-based Maximum-of-difference Test for Two-sample Comparison (ICML 2026 · similarity 0.85)
- On the Hardness of Conditional Independence Testing In Practice (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
