Research map / Label-efficient and robust learning / Test-time adaptation
AutoML Two-Sample Test
Jonas M. Kübler, Vincent Stimper, Simon Buchholz, Krikamol Muandet, Bernhard Schölkopf
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use an AutoML-trained continuous witness on raw features | Two-sample shift detection · MNIST and CIFAR10 distribution shifts, 10,000 test samples | test power: 0.87 vs 0.79 | Improved |
| Threshold classifier probabilities to binary outputs (ablation) | AutoML witness test · MNIST and CIFAR10 distribution shifts, 10,000 test samples | test power: 0.87 -> 0.86 | Hurt |
| Use classifier probabilities as a continuous witness rather than binary predictions (ablation) | AutoML witness test · Blob, sample size 180 | test power: 0.39±0.02 -> 0.54±0.02 | Improved |
| Train on pretrained softmax features rather than raw inputs (ablation) | AutoML witness test · MNIST and CIFAR10 distribution shifts, 20 test samples | test power: 0.24 -> 0.29 | Improved |
| Train on pretrained softmax features rather than raw inputs (ablation) | AutoML witness test · MNIST and CIFAR10 distribution shifts, 10,000 test samples | test power: 0.87 -> 0.72 | Hurt |
| Use pretrained features instead of raw inputs (ablation) | AutoML witness test · Small and medium Gaussian-noise shifts | test power: s_gn, 10,000 samples: 0.80 -> 0.03 | Hurt |
| Use pretrained features instead of raw inputs (ablation) | AutoML witness test · Knock-out shifts | test power: ko, 20 samples: 0.33 -> 0.47 | Improved |
| Increase training time limit from 1 to 10 minutes (ablation) | AutoML witness test · Blob, sample size 180 | test power: 0.56±0.02 -> 0.56±0.02 | No clear effect |
| Increase training time limit from 1 to 10 minutes (ablation) | AutoML witness test · Higgs, sample size 1000 | test power: 0.13±0.02 -> 0.09±0.01 | Hurt |
| Increase training time limit from 1 to 10 minutes (ablation) | AutoML witness test · Higgs, sample size 10000 | test power: 0.94±0.01 -> 0.99±0.00 | Improved |
| Optimize the witness with MSE and use its continuous mean discrepancy | AutoML two-sample test · Blob | test power | Improved |
| Use AutoML rather than a fixed-architecture binary classifier | Distribution-shift detection · MNIST and CIFAR10 distribution shifts | test power | Improved |
Closest papers by meaning
- Optimal Algorithms for Augmented Testing of Discrete Distributions (NeurIPS 2024 · similarity 0.84)
- Kernel Selection is Model Selection: A Unified Complexity-Penalised Approach for MMD Two-Sample Tests (NeurIPS 2026 · similarity 0.83)
- Sequential Kernel Goodness-of-fit Testing (ICML 2024 · similarity 0.82)
- YEAST: Yet Another Sequential Test (NeurIPS 2025 · similarity 0.82)
- Strategic A/B testing via Maximum Probability-driven Two-armed Bandit (ICML 2025 · similarity 0.82)
- DUAL: Learning Diverse Kernels for Aggregated Two-sample and Independence Testing (NeurIPS 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
