Research map / Probabilistic and scientific modeling / Variational and generative sampling
Distribution Free Tests for Model Selection Based on Maximum Mean Discrepancy with Estimated Parameters
Florian Brück, Jean-David Fermanian, Aleksey Min
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use the corrected test statistic Tn(M,P) with estimated parameters instead of sqrt(n) times plain MMD (ablation) | MMD specification test · Gaussian simulations, n = 500, p in {2,16} | empirical power: always higher than the test solely based on sqrt(n) MMD | Improved |
| let epsilon_n decay more slowly (n^-1/6.5 instead of n^-1/2.5) (ablation) | Tn(M,P) specification test · Gaussian simulations, n = 500 | empirical power: epsilon_n in {n^-1/2.5, n^-1/4.5, n^-1/6.5}; power decreasing as convergence rate decreases | Hurt |
| use the corrected statistic Tn(M1,M2,P) for model comparison instead of the plain MMD difference statistic (ablation) | MMD model comparison test · Degenerate case, n in {500,1000} | empirical power: always outperforms the competitor test statistic for n in {500,1000} | Improved |
| use the corrected statistic Tn(M1,M2,P) for model comparison in the non-degenerate case (ablation) | MMD model comparison test · Non-degenerate case, p in {2,16} | empirical power: more powerful than a test based on sqrt(n) MMD difference | Improved |
| increase sample size from n = 100 | MMD specification tests · Gaussian simulations, p = 2 | empirical power: poor power of our MMD specification tests for a small sample size (n = 100) and a small dimension (d = 2) | Mixed |
| increase dimension from 2 to 16 | MMD tests · Gaussian simulations | empirical power: p in {2,16}; power increases with increasing dimension | Improved |
| replace normal copula with Clayton copula under t margins | seven-dimensional MSCI residual model · MSCI residuals specification test | p-value: 0.8444 -> 1.9 ⋅10−29 | Hurt |
| choose Gumbel copula over Clayton copula | seven-dimensional MSCI residual model · MSCI residuals model comparison | test statistic: test statistic 3.4271, p-value 0.0006 | Improved |
Closest papers by meaning
- Kernel Selection is Model Selection: A Unified Complexity-Penalised Approach for MMD Two-Sample Tests (NeurIPS 2026 · similarity 0.86)
- Hypothesis Testing for Generalized Thurstone Models (ICML 2025 · similarity 0.83)
- Post-Selection Distributional Model Evaluation (NeurIPS 2026 · similarity 0.83)
- The Polynomial Stein Discrepancy for Assessing Moment Convergence (ICML 2025 · similarity 0.82)
- Revealing Distribution Discrepancy by Sampling Transfer in Unlabeled Data (NeurIPS 2024 · similarity 0.81)
- Asymptotically Log-Optimal Bayes-Assisted Confidence Sequences for Bounded Means (NeurIPS 2026 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
