Research map / Learning theory and algorithms / Kernels and regression theory
Efficient Statistical Tests: A Neural Tangent Kernel Approach
Sheng Jia, Ehsan Nezhadarya, Yuhuai Wu, Jimmy Ba
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use shift-invariant cosine first layer (SCNTK) instead of ReLU CNTK (ablation) | NTK kernel for MMD two-sample tests · MNIST DCGAN shift (Table 3) | successful rejection rate (avg): 0.807 vs 0.782 | Improved |
| use shift-invariant cosine first layer (SCNTK) instead of ReLU CNTK (ablation) | NTK kernel for MMD two-sample tests · CIFAR10 vs CIFAR10.1 (Table 3) | successful rejection rate: 0.805 vs 0.742 | Improved |
| use SCNTK random-feature approximation instead of exact NTK or Monte-Carlo NTK (ablation) | SCNTK · MNIST DCGAN shift (Table 3) | successful rejection rate (avg): 0.807 vs 0.814 exact, 0.809 MC | No clear effect |
| replace shift-invariant NTK kernel with naive Laplace kernel in pixel space (ablation) | two-sample test kernel · MNIST DCGAN shift (Table 3) | successful rejection rate (avg): 0.437 vs 0.807 | Hurt |
| increase network channel width from 100 to 300 (ablation) | SCNTK for MNIST two-sample test · MNIST (Table 1) | empirical average test power: 0.18 to 0.81 | Improved |
| use shift-invariant SCNTK KDE instead of ReLU CNTK KDE for outlier detection (ablation) | KDE-based OOD detection · CIFAR10 vs SVHN (Table 5) | AUROC: 0.85 vs 0.71 (CIFAR10 inlier); 0.80 vs 0.51 (SVHN inlier) | Improved |
| fixed SCNTK kernel (no kernel training) vs optimized deep kernel MMD-D on MNIST DCGAN shift | MMD two-sample test · MNIST DCGAN shift (Table 2) | successful rejection rate (avg): 0.807 vs 0.91 for MMD-D | Hurt |
| fixed SCNTK kernel vs optimized deep kernel MMD-D on CIFAR10 vs CIFAR10.1 | MMD two-sample test · CIFAR10 vs CIFAR10.1 (Table 2) | successful rejection rate: 0.805 vs 0.744 | Improved |
| fixed SCNTK kernel vs optimized bandwidth Gaussian MMD-O | MMD two-sample test · MNIST and CIFAR10 (Table 2) | successful rejection rate: MNIST avg 0.807 vs 0.572; CIFAR 0.805 vs 0.316 | Improved |
| use shift-invariant random-feature Gaussian-style first layer and gradient-based kernel vs fixed Gaussian kernel on untrained autoencoder features under dataset shifts | MMD two-sample tests on MNIST/CIFAR10 with injected shifts · Gaussian noise, rotation/translation, adversarial shifts (Figure 2) | dataset shift detection accuracy: more than 2x improvement reported | Improved |
| fixed SCNTK on class-imbalance shift vs MMD-UAE baseline | MMD two-sample tests · imbalanced dataset shift (Figure 2) | dataset shift detection accuracy | No clear effect |
Closest papers by meaning
- Learning Deep Kernels for Non-Parametric Two-Sample Tests (ICML 2020 · similarity 0.87)
- Neural Tangent Kernel Maximum Mean Discrepancy (NeurIPS 2021 · similarity 0.86)
- KS(conf): A Light-Weight Test if a Multiclass Classifier Operates Outside of Its Specifications (IJCV 2019 · similarity 0.86)
- The Neural Tangent Kernel in High Dimensions: Triple Descent and a Multi-Scale Theory of Generalization (ICML 2020 · similarity 0.85)
- RODEO: Robust Outlier Detection via Exposing Adaptive Out-of-Distribution Samples (ICML 2024 · similarity 0.83)
- Expecting The Unexpected: Towards Broad Out-Of-Distribution Detection (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
