AI Paper Atlas

Research map / Label-efficient and robust learning / Test-time adaptation

AutoML Two-Sample Test

Jonas M. Kübler, Vincent Stimper, Simon Buchholz, Krikamol Muandet, Bernhard Schölkopf

NeurIPS 2022 · Label-efficient and robust learning › Test-time adaptation · approach: Robust optimization for distribution shift · 1 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use an AutoML-trained continuous witness on raw featuresTwo-sample shift detection · MNIST and CIFAR10 distribution shifts, 10,000 test samplestest power: 0.87 vs 0.79Improved
Threshold classifier probabilities to binary outputs (ablation)AutoML witness test · MNIST and CIFAR10 distribution shifts, 10,000 test samplestest power: 0.87 -> 0.86Hurt
Use classifier probabilities as a continuous witness rather than binary predictions (ablation)AutoML witness test · Blob, sample size 180test power: 0.39±0.02 -> 0.54±0.02Improved
Train on pretrained softmax features rather than raw inputs (ablation)AutoML witness test · MNIST and CIFAR10 distribution shifts, 20 test samplestest power: 0.24 -> 0.29Improved
Train on pretrained softmax features rather than raw inputs (ablation)AutoML witness test · MNIST and CIFAR10 distribution shifts, 10,000 test samplestest power: 0.87 -> 0.72Hurt
Use pretrained features instead of raw inputs (ablation)AutoML witness test · Small and medium Gaussian-noise shiftstest power: s_gn, 10,000 samples: 0.80 -> 0.03Hurt
Use pretrained features instead of raw inputs (ablation)AutoML witness test · Knock-out shiftstest power: ko, 20 samples: 0.33 -> 0.47Improved
Increase training time limit from 1 to 10 minutes (ablation)AutoML witness test · Blob, sample size 180test power: 0.56±0.02 -> 0.56±0.02No clear effect
Increase training time limit from 1 to 10 minutes (ablation)AutoML witness test · Higgs, sample size 1000test power: 0.13±0.02 -> 0.09±0.01Hurt
Increase training time limit from 1 to 10 minutes (ablation)AutoML witness test · Higgs, sample size 10000test power: 0.94±0.01 -> 0.99±0.00Improved
Optimize the witness with MSE and use its continuous mean discrepancyAutoML two-sample test · Blobtest powerImproved
Use AutoML rather than a fixed-architecture binary classifierDistribution-shift detection · MNIST and CIFAR10 distribution shiftstest powerImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy